Oct 29 23:10:33.342673 ip-10-0-141-12 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 29 23:10:33.342686 ip-10-0-141-12 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 29 23:10:33.342695 ip-10-0-141-12 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 29 23:10:33.343131 ip-10-0-141-12 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 29 23:10:42.547803 ip-10-0-141-12 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 29 23:10:42.547814 ip-10-0-141-12 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1b67e816e9b549fead83890f187f3cd2 --
Oct 29 23:12:27.092237 ip-10-0-141-12 systemd[1]: Starting Kubernetes Kubelet...
Oct 29 23:12:27.468620 ip-10-0-141-12 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.468620 ip-10-0-141-12 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 29 23:12:27.468620 ip-10-0-141-12 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.468620 ip-10-0-141-12 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:12:27.476368 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476306    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.476368 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476367    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.476368 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476371    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476374    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476377    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476379    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476382    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476385    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476387    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476390    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476392    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476395    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476400    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476404    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476407    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476410    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476412    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476415    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476418    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476421    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476431    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476435    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.476459 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476438    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476441    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476444    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476446    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476449    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476451    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476454    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476456    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476459    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476462    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476464    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476467    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476470    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476473    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476476    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476478    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476481    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476484    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476486    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.476945 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476489    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476491    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476494    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476496    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476498    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476501    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476504    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476506    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476509    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476511    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476513    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476516    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476522    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476525    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476528    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476531    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476534    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476536    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476539    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.477411 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476541    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476544    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476546    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476549    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476552    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476554    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476557    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476560    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476562    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476565    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476567    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476570    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476573    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476575    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476578    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476580    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476583    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476585    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476589    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476591    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.477875 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476594    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476596    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476598    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476601    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476603    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476606    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476609    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476612    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476616    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476619    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476622    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476624    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476627    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476629    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476632    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476635    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476638    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476640    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476643    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.478392 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476645    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476648    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476651    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476653    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476657    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476659    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.476662    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478440    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478447    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478451    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478454    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478457    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478461    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478465    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478469    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478472    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478474    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478477    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478480    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478483    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.478833 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478486    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478488    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478491    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478494    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478496    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478499    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478502    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478504    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478507    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478509    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478511    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478514    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478516    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478519    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478522    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478525    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478528    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478530    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478533    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478536    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.479327 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478538    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478541    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478543    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478546    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478548    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478551    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478553    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478556    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478558    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478561    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478563    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478566    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478568    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478571    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478574    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478576    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478579    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478581    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478584    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478586    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.479801 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478589    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478592    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478595    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478597    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478600    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478602    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478605    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478608    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478611    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478614    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478616    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478619    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478622    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478624    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478627    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478629    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478632    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478634    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478637    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.480301 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478639    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478642    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478644    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478647    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478650    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478652    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478654    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478657    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478660    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478663    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478665    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478668    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478670    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478673    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478675    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478678    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478680    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478683    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478685    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478688    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.480763 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478691    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478694    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478697    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478701    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478704    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478707    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478710    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478713    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478715    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478718    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478720    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478723    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478727    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.478729    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479204    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479209    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479212    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479215    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479218    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479221    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.481250 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479223    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479226    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479229    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479231    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479234    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479237    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479239    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479242    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479246    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479249    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479253    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479258    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479261    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479264    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479267    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479269    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479272    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479275    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479278    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.481733 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479280    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479283    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479285    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479288    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479291    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479293    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479296    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479298    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479301    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479304    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479306    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479309    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479311    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479314    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479317    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479320    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479322    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479325    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479328    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479330    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.482215 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479334    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479336    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479339    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479341    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479344    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479347    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479349    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479352    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479354    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479357    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479360    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479362    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479365    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479367    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479370    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479373    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479375    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479378    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479396    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479401    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.482673 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479404    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479408    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479411    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479415    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479419    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479425    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479428    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479431    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479434    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479437    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479439    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479442    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479444    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479447    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479450    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479452    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479455    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479457    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479460    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.483144 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479462    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479465    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479468    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479471    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479473    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479476    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479479    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479481    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479484    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479486    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479489    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479491    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479494    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479497    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479499    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479502    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479504    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479507    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479511    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.483626 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479515    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479518    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.479521    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481458    2586 flags.go:64] FLAG: --address="0.0.0.0"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481472    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481479    2586 flags.go:64] FLAG: --anonymous-auth="true"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481484    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481488    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481492    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481496    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481500    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481504    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481507    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481511    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481514    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481517    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481520    2586 flags.go:64] FLAG: --cgroup-root=""
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481523    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481526    2586 flags.go:64] FLAG: --client-ca-file=""
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481529    2586 flags.go:64] FLAG: --cloud-provider="external"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481532    2586 flags.go:64] FLAG: --cluster-dns="[]"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481537    2586 flags.go:64] FLAG: --cluster-domain=""
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481540    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481543    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 29 23:12:27.484921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481547    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481550    2586 flags.go:64] FLAG: --container-log-max-files="5"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481554    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481557    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481560    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481564    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481567    2586 flags.go:64] FLAG: --contention-profiling="false"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481570    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481573    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481577    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481581    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481586    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481589    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481593    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481595    2586 flags.go:64] FLAG: --enable-load-reader="false"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481598    2586 flags.go:64] FLAG: --enable-server="true"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481601    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481605    2586 flags.go:64] FLAG: --event-burst="100"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481608    2586 flags.go:64] FLAG: --event-qps="50"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481611    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481614    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481617    2586 flags.go:64] FLAG: --eviction-hard=""
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481621    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481624    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481627    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 29 23:12:27.485528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481630    2586 flags.go:64] FLAG: --eviction-soft=""
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481633    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481635    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481639    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481641    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481644    2586 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481647    2586 flags.go:64] FLAG: --fail-swap-on="true"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481650    2586 flags.go:64] FLAG: --feature-gates=""
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481654    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481658    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481661    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481664    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481667    2586 flags.go:64] FLAG: --healthz-port="10248"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481670    2586 flags.go:64] FLAG: --help="false"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481673    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481677    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481680    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481682    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481687    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481691    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481694    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481697    2586 flags.go:64] FLAG: --image-service-endpoint=""
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481700    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 29 23:12:27.486150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481702    2586 flags.go:64] FLAG: --kube-api-burst="100"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481706    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481709    2586 flags.go:64] FLAG: --kube-api-qps="50"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481712    2586 flags.go:64] FLAG: --kube-reserved=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481714    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481717    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481721    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481724    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481726    2586 flags.go:64] FLAG: --lock-file=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481740    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481743    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481747    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481752    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481755    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481758    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481761    2586 flags.go:64] FLAG: --logging-format="text"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481764    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481767    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481770    2586 flags.go:64] FLAG: --manifest-url=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481773    2586 flags.go:64] FLAG: --manifest-url-header=""
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481777    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481780    2586 flags.go:64] FLAG: --max-open-files="1000000"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481784    2586 flags.go:64] FLAG: --max-pods="110"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481787    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481790    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 29 23:12:27.486709 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481793    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481796    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481799    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481803    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481806    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481813    2586 flags.go:64] FLAG: --node-status-max-images="50"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481816    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481819    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481822    2586 flags.go:64] FLAG: --pod-cidr=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481825    2586 flags.go:64] FLAG: --pod-manifest-path=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481828    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481831    2586 flags.go:64] FLAG: --pods-per-core="0"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481834    2586 flags.go:64] FLAG: --port="10250"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481837    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481839    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab0571271bd57984"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481842    2586 flags.go:64] FLAG: --qos-reserved=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481845    2586 flags.go:64] FLAG: --read-only-port="10255"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481848    2586 flags.go:64] FLAG: --register-node="true"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481852    2586 flags.go:64] FLAG: --register-with-taints=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481855    2586 flags.go:64] FLAG: --registry-burst="10"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481858    2586 flags.go:64] FLAG: --registry-qps="5"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481861    2586 flags.go:64] FLAG: --reserved-cpus=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481864    2586 flags.go:64] FLAG: --reserved-memory=""
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481868    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481871    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481874    2586 flags.go:64] FLAG: --rotate-certificates="false"
Oct 29 23:12:27.487314 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481877    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481880    2586 flags.go:64] FLAG: --runonce="false"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481883    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481886    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481889    2586 flags.go:64] FLAG: --seccomp-default="false"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481891    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481894    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481897    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481900    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481903    2586 flags.go:64] FLAG: --storage-driver-password="root"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481907    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481909    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481912    2586 flags.go:64] FLAG: --storage-driver-user="root"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481915    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481918    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481921    2586 flags.go:64] FLAG: --system-cgroups=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481924    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481929    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481932    2586 flags.go:64] FLAG: --tls-cert-file=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481935    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481939    2586 flags.go:64] FLAG: --tls-min-version=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481942    2586 flags.go:64] FLAG: --tls-private-key-file=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481945    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481948    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481950    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 29 23:12:27.488032 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481953    2586 flags.go:64] FLAG: --v="2"
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481957    2586 flags.go:64] FLAG: --version="false"
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481962    2586 flags.go:64] FLAG: --vmodule=""
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481966    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.481969    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482108    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482113    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482116    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482121    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482124    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482127    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482129    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482132    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482135    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482138    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482140    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482143    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482146    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482149    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482152    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482154    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.488621 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482157    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482160    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482162    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482165    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482168    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482170    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482174    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482178    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482182    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482189    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482192    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482195    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482197    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482200    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482203    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482205    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482207    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482210    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482214    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.489261 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482218    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482221    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482224    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482227    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482229    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482232    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482235    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482237    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482240    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482242    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482245    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482248    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482250    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482253    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482256    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482258    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482261    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482263    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482266    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.490034 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482268    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482271    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482273    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482276    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482278    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482281    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482283    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482286    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482288    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482291    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482293    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482296    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482300    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482303    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482306    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482308    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482311    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482313    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482316    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.490569 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482319    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482321    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482324    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482326    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482329    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482332    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482334    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482339    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482342    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482345    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482348    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482351    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482353    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482356    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482358    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482361    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482363    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482366    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482368    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.491046 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482371    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482373    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482376    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482378    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482381    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482383    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482387    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482391    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482393    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482396    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482398    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482401    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482404    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482406    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.482431    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482471    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.491487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482475    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482478    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482481    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482483    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482486    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482489    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482492    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482494    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482497    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482499    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482502    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482504    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482507    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482509    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482512    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482515    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482517    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482520    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482522    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482525    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.491881 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482527    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482530    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482533    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482536    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482540    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482542    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482545    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482547    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482550    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482553    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482555    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482558    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482560    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482563    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482566    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482568    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482571    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482575    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482579    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482582    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.492378 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482585    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482587    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482590    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482592    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482595    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482597    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482600    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482602    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482605    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482608    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482610    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482613    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482616    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482618    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482622    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482624    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482628    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482631    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482634    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.492844 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482636    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482639    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482641    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482644    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482647    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482650    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482652    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482655    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482657    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482660    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482662    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482665    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482668    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482670    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482673    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482676    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482678    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482681    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482684    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.493324 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482686    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482689    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482691    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482694    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482696    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482699    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482701    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482704    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482707    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482710    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482714    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482716    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482719    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482721    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482724    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482726    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482729    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482733    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482737    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.493768 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482740    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482743    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482746    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482749    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482752    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482754    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482757    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.482760    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.491265    2586 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.491279    2586 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.491302    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.491307    2586 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491363    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491368    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491372    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491375    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491377    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491380    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491383    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.494236 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491386    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491388    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491391    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491394    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491396    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491399    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491401    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491404    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491408    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491410    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491413    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491415    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491418    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491421    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491423    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491426    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491428    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491431    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491433    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491436    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.494704 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491438    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491441    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491444    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491447    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491450    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491452    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491455    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491458    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491460    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491463    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491465    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491468    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491470    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491473    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491476    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491478    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491481    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491483    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491486    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491489    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.495305 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491492    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491495    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491497    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491500    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491503    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491506    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491508    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491511    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491513    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491516    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491519    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491521    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491524    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491526    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491530    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491532    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491535    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491538    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491540    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.495762 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491543    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491545    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491548    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491552    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491556    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491559    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491562    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491564    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491567    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491570    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491573    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491576    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491579    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491582    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491585    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491588    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491590    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491593    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491596    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.496260 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491598    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491601    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491603    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491606    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491608    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491611    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491613    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491617    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491621    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491624    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491627    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491630    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491632    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491635    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491637    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491640    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491642    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491645    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491648    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.496694 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491650    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491653    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.491679    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491810    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491816    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491820    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491823    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491826    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491830    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491832    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491835    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491838    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491841    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491844    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491855    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491858    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.497160 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491861    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491863    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491866    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491869    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491871    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491874    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491876    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491879    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491881    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491884    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491887    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491889    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491892    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491895    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491897    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491900    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491902    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491905    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491907    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.497536 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491910    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491912    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491916    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491921    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491925    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491928    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491933    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491937    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491940    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491943    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491945    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491948    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491951    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491953    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491956    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491958    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491961    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491963    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491966    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.498004 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491968    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491985    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491989    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491992    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491995    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491997    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.491999    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492002    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492005    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492008    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492010    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492013    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492015    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492018    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492020    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492023    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492026    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492028    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492031    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492033    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.498510 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492037    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492039    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492042    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492045    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492047    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492049    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492052    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492054    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492057    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492060    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492062    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492065    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492067    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492070    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492072    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492075    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492077    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492080    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492082    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492085    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.499007 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492087    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492089    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492092    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492095    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492097    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492100    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492102    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492105    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492108    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492111    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492114    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492116    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492119    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492121    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492124    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.499487 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.492150    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492183    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492187    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492191    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492195    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492198    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492200    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492203    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492205    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492208    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492210    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492213    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492215    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492218    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492221    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492224    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492226    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492229    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492231    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:12:27.499862 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492234    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492236    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492239    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492242    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492244    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492247    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492249    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492252    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492254    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492257    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492259    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492262    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492265    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492268    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492270    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492273    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492275    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492278    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492280    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492282    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:12:27.500462 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492285    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492287    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492290    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492293    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492297    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492299    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492302    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492306    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492309    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492312    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492315    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492318    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492323    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492326    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492328    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492332    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492334    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492337    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492340    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:12:27.500949 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492342    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492345    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492347    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492350    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492352    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492355    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492358    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492360    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492363    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492366    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492368    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492371    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492373    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492376    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492378    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492383    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492386    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492388    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492391    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492394    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:12:27.501526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492396    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492399    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492401    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492404    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492406    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492410    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492413    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492415    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492418    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492420    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492423    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492426    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492429    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492431    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492434    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492437    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492439    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492442    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492444    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492451    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:12:27.502114 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492454    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492457    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492460    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492462    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492465    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492467    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492470    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492474    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:27.492477    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.492526    2586 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.494749    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.495818    2586 server.go:1014] "Starting client certificate rotation"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.495908    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.495947    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.498187    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:12:27.502634 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.499918    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:12:27.511710 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.511694    2586 log.go:25] "Validated CRI v1 runtime API"
Oct 29 23:12:27.515462 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.515449    2586 log.go:25] "Validated CRI v1 image API"
Oct 29 23:12:27.516306 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.516282    2586 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 29 23:12:27.522921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.522898    2586 fs.go:135] Filesystem UUIDs: map[24fb1fde-7b0c-4328-afa4-cd4cd3b36d3c:/dev/nvme0n1p3 3cfa7feb-b3bb-47df-8bee-348f92be40fe:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 29 23:12:27.523009 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.522919    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 29 23:12:27.528787 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.528665    2586 manager.go:217] Machine: {Timestamp:2026-10-29 23:12:27.527160276 +0000 UTC m=+0.329155282 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099004 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec226a6734fb29efdb5b6c4ccfdc1a69 SystemUUID:ec226a67-34fb-29ef-db5b-6c4ccfdc1a69 BootID:1b67e816-e9b5-49fe-ad83-890f187f3cd2 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f8:d0:72:1f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f8:d0:72:1f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:cb:8e:bb:e8:f0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 29 23:12:27.528787 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.528779    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 29 23:12:27.528947 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.528903    2586 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 29 23:12:27.533108 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533080    2586 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 29 23:12:27.533234 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533107    2586 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 29 23:12:27.533903 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533893    2586 topology_manager.go:143] "Creating topology manager with none policy"
Oct 29 23:12:27.533935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533905    2586 container_manager_linux.go:311] "Creating device plugin manager"
Oct 29 23:12:27.533935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533918    2586 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:12:27.533935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.533928    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:12:27.535103 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.535092    2586 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 29 23:12:27.539685 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.539671    2586 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 29 23:12:27.539792 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.539783    2586 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 29 23:12:27.540525 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540515    2586 kubelet.go:493] "Attempting to sync node with API server"
Oct 29 23:12:27.540566 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540527    2586 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 29 23:12:27.540566 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540537    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 29 23:12:27.540566 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540546    2586 kubelet.go:405] "Adding apiserver pod source"
Oct 29 23:12:27.540566 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540553    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 29 23:12:27.540833 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.540820    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:12:27.543459 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.543440    2586 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 29 23:12:27.549992 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.549963    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:12:27.550925 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.550911    2586 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 29 23:12:27.551000 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.550941    2586 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 29 23:12:27.551114 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551103    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 29 23:12:27.551142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551118    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 29 23:12:27.551142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551125    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 29 23:12:27.551142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551131    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 29 23:12:27.551142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551137    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 29 23:12:27.551142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551143    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 29 23:12:27.551876 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551867    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 29 23:12:27.551905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551877    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 29 23:12:27.551905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551884    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 29 23:12:27.551905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551890    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 29 23:12:27.551905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551898    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 29 23:12:27.551905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.551905    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 29 23:12:27.552770 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.552761    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 29 23:12:27.552770 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.552771    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 29 23:12:27.556432 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.556420    2586 server.go:1263] "Started kubelet"
Oct 29 23:12:27.556534 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.556507    2586 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 29 23:12:27.556588 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.556497    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 29 23:12:27.556588 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.556575    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 29 23:12:27.557080 ip-10-0-141-12 systemd[1]: Started Kubernetes Kubelet.
Oct 29 23:12:27.558292 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.558053    2586 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 29 23:12:27.558742 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.558727    2586 server.go:317] "Adding debug handlers to kubelet server"
Oct 29 23:12:27.564674 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.564642    2586 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 29 23:12:27.566110 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.566089    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 29 23:12:27.566861 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.566849    2586 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 29 23:12:27.566958 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.566939    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:27.567630 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.567603    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 29 23:12:27.567830 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.567615    2586 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 29 23:12:27.567911 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.567834    2586 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 29 23:12:27.568002 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.567988    2586 reconstruct.go:97] "Volume reconstruction finished"
Oct 29 23:12:27.568053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568003    2586 reconciler.go:29] "Reconciler: start to sync state"
Oct 29 23:12:27.568118 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568099    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 29 23:12:27.568159 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568120    2586 factory.go:55] Registering systemd factory
Oct 29 23:12:27.568159 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568126    2586 factory.go:223] Registration of the systemd container factory successfully
Oct 29 23:12:27.568673 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568659    2586 factory.go:153] Registering CRI-O factory
Oct 29 23:12:27.568728 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568699    2586 factory.go:223] Registration of the crio container factory successfully
Oct 29 23:12:27.568728 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568717    2586 factory.go:103] Registering Raw factory
Oct 29 23:12:27.568817 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.568730    2586 manager.go:1199] Started watching for new ooms in manager
Oct 29 23:12:27.569813 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.569799    2586 manager.go:319] Starting recovery of all containers
Oct 29 23:12:27.581042 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.581000    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 29 23:12:27.581123 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.581013    2586 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 29 23:12:27.582264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.582247    2586 manager.go:324] Recovery completed
Oct 29 23:12:27.584715 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.584689    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 29 23:12:27.587547 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.587533    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.589635 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.589617    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.589712 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.589648    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.589712 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.589661    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.590217 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590197    2586 cpu_manager.go:226] "Starting" policy="none"
Oct 29 23:12:27.590217 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590211    2586 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 29 23:12:27.590373 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590225    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.590373 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590229    2586 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 29 23:12:27.590932 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590917    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.591016 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590946    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.591016 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.590961    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.591317 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.590173    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-12.ec2.internal.18e322fd3648b984  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-12.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-12.ec2.internal,},FirstTimestamp:2026-10-29 23:12:27.556395396 +0000 UTC m=+0.358390402,LastTimestamp:2026-10-29 23:12:27.556395396 +0000 UTC m=+0.358390402,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-12.ec2.internal,}"
Oct 29 23:12:27.592531 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.592519    2586 policy_none.go:50] "Start"
Oct 29 23:12:27.592568 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.592535    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 29 23:12:27.592568 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.592545    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 29 23:12:27.594686 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.594676    2586 policy_none.go:44] "Start"
Oct 29 23:12:27.603815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.603787    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 29 23:12:27.603815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.603785    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 29 23:12:27.606677 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.606659    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtgm8"
Oct 29 23:12:27.615123 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.615108    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtgm8"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.625559    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.625592    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-12.ec2.internal.18e322fd3843e73f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-12.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-12.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-12.ec2.internal,},FirstTimestamp:2026-10-29 23:12:27.589633855 +0000 UTC m=+0.391628861,LastTimestamp:2026-10-29 23:12:27.589633855 +0000 UTC m=+0.391628861,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-12.ec2.internal,}"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.632687    2586 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.633888    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.635052    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.635674    2586 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.635707    2586 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.635747    2586 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.636244    2586 manager.go:341] "Starting Device Plugin manager"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.636989    2586 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637004    2586 server.go:87] "Starting device plugin registration server"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637229    2586 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637240    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637330    2586 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637410    2586 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.637420    2586 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.637913    2586 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 29 23:12:27.644805 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.637965    2586 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:27.647819 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.647802    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:27.736895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.736841    2586 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"]
Oct 29 23:12:27.736895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.736894    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.737369 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.737355    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.737641 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.737626    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.737703 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.737654    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.737703 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.737669    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.737986 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.737957    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.738033 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.738004    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.738033 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.738014    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.738033 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.738033    2586 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.739683 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.739671    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.739817 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.739805    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.739864 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.739832    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.740377 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740358    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.740470 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740358    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.740470 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740411    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.740470 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740424    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.740470 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740390    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.740470 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.740461    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.742506 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.742488    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.742574 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.742522    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:27.743210 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.743193    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:27.743305 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.743221    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:27.743305 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.743231    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:27.754464 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.754448    2586 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.754540 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.754469    2586 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-12.ec2.internal\": node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:27.769124 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.769105    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.769197 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.769132    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.769244 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.769207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.769283 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.769245    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-12.ec2.internal\" not found" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.772457 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.772437    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-12.ec2.internal\" not found" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.792679 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.792662    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:27.869508 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869486    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.869606 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869510    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.869606 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869527    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.869606 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869570    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.869731 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869605    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.869731 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:27.869603    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:27.893706 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.893683    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:27.994821 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:27.994767    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.073748 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.073723    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:28.076884 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.076865    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:28.095495 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.095473    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.196048 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.196021    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.296616 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.296561    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.397112 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.397089    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.495650 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.495627    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 29 23:12:28.496106 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.495788    2586 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 29 23:12:28.497781 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.497764    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.598339 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:28.598165    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:12:28.625082 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.625007    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:27 +0000 UTC" deadline="2028-04-28 02:38:30.820222002 +0000 UTC"
Oct 29 23:12:28.625082 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.625069    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13107h26m2.195155708s"
Oct 29 23:12:28.632863 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.632844    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:12:28.634593 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.634575    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:12:28.634698 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.634613    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:12:28.634698 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.634627    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:12:28.674349 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.674328    2586 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:28.716300 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:28.716272    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad8bdceb419b718ac0187fcaf3ee427.slice/crio-3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc WatchSource:0}: Error finding container 3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc: Status 404 returned error can't find the container with id 3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc
Oct 29 23:12:28.716526 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:28.716503    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc940380ed911d9e6c421c48f4b1a6a.slice/crio-04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef WatchSource:0}: Error finding container 04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef: Status 404 returned error can't find the container with id 04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef
Oct 29 23:12:28.721577 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.721561    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:12:28.733687 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.733669    2586 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:28.767492 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.767469    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:28.790124 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.790105    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:12:28.792179 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.792166    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:12:28.816257 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:28.816237    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:12:29.114412 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.114387    2586 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:29.541843 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.541766    2586 apiserver.go:52] "Watching apiserver"
Oct 29 23:12:29.568160 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.568126    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 29 23:12:29.585818 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.585788    2586 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 29 23:12:29.586152 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.586133    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-l9kgz","openshift-network-diagnostics/network-check-target-q9xkk","kube-system/global-pull-secret-syncer-64m88","openshift-cluster-node-tuning-operator/tuned-lhps4","openshift-network-operator/iptables-alerter-nwtsk","openshift-ovn-kubernetes/ovnkube-node-jdnh8","kube-system/konnectivity-agent-b9h9g","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal","openshift-multus/multus-additional-cni-plugins-7g7tb","openshift-multus/network-metrics-daemon-vqlgv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk","openshift-dns/node-resolver-4blvf","kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal","openshift-image-registry/node-ca-g9m5t"]
Oct 29 23:12:29.588819 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.588792    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.588920 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.588888    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:29.591364 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.590836    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.593143 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.593117    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.595518 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.595499    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.597940 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.597922    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.600082 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.600064    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.602318 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.602296    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.605271 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.605151    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:29.605271 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.605212    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:29.607533 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.607513    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.609566 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.609543    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 29 23:12:29.609893 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.609871    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qbkzf\""
Oct 29 23:12:29.610013 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.610000    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.610303 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.610285    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.610385 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.610304    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7zkzr\""
Oct 29 23:12:29.610568 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.610553    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.611356 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.611340    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 29 23:12:29.611590 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.611570    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.611682 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.611667    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.613517 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.612218    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.614578 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.614556    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.614669 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.614620    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:29.616011 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.615991    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 29 23:12:29.616108 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.616049    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5wtxz"
Oct 29 23:12:29.641902 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.641855    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal" event={"ID":"9bc940380ed911d9e6c421c48f4b1a6a","Type":"ContainerStarted","Data":"04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef"}
Oct 29 23:12:29.643538 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.643517    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerStarted","Data":"3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc"}
Oct 29 23:12:29.646312 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.646288    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:12:29.669197 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.669173    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 29 23:12:29.680674 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680643    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.680785 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680680    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.680785 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680706    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.680785 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680730    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvs8h\" (UniqueName: \"kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.680905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680824    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.680905 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680884    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.680967 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680918    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.680967 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680936    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.680967 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680950    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.681114 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.680968    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.681114 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681030    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.681114 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681068    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.681114 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681095    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681136    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681166    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681196    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681221    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681244    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bkm9\" (UniqueName: \"kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.681277 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681270    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681321    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681342    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681374    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681393    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681412    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681447    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681478    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:29.681503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681497    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681549    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681583    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681610    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681632    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681659    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681675    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681698    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681712    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.681751 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681731    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681751    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss25j\" (UniqueName: \"kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681771    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681784    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681808    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681836    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681852    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681867    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.681961    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682001    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682031    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682047    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682063    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682081 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682077    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g259q\" (UniqueName: \"kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682095    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682115    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682130    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682148    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682168    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682188    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682219    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz42q\" (UniqueName: \"kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682238    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682253    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682279    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682300    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g6mc\" (UniqueName: \"kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682315    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682330    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682351    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682368    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwlfl\" (UniqueName: \"kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.682597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682386    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682411    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmdn8\" (UniqueName: \"kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682435    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682458    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682473    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682488    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682502    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682515    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682642    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682661    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9grgn\" (UniqueName: \"kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682677    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682691    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682711    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682730    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682744    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682758    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.683203 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.682775    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.709329 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709296    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.709329 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709336    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.709535 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709427    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 29 23:12:29.709535 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709482    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8km5l\""
Oct 29 23:12:29.709535 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709490    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 29 23:12:29.709535 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709533    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.709716 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709650    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-pgvzg\""
Oct 29 23:12:29.709716 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709693    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.709716 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709693    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 29 23:12:29.709851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709728    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.709851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709735    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.709851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709748    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 29 23:12:29.709851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709745    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.709851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709808    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-t8tv9\""
Oct 29 23:12:29.710092 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709856    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.710092 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.709893    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pnrbh\""
Oct 29 23:12:29.723024 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723000    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 29 23:12:29.723160 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723031    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 29 23:12:29.723450 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723430    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 29 23:12:29.723538 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723475    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 29 23:12:29.723538 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723434    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 29 23:12:29.723538 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723496    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9w58g\""
Oct 29 23:12:29.723538 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723506    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:29.723733 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723618    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hwcbb\""
Oct 29 23:12:29.723733 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723633    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 29 23:12:29.723733 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723664    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jb6zf\""
Oct 29 23:12:29.723733 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723635    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 29 23:12:29.723913 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.723894    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 29 23:12:29.784025 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.783988    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvs8h\" (UniqueName: \"kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784032    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.784181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784060    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.784181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784088    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784134    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.784181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784158    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784185    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784211    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784237    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784268    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784291    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784313    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.784344 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784336    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784359    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784382    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bkm9\" (UniqueName: \"kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784406    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784431    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784447    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784460    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784474    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784489    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784503    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784540 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784518    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784555    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784570    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784586    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784616    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784632    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784653    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784667    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784684    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784700    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784715    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss25j\" (UniqueName: \"kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784732    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784746    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784760    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784774    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.784796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784788    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784801    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784817    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784831    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784846    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784860    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784873    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784888    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g259q\" (UniqueName: \"kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784903    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784918    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784934    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784964    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.784992    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785018    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz42q\" (UniqueName: \"kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785062    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785088    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.785360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785093    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785112    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785122    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785158    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785161    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785172    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785218    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785224    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785219    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785251    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785277    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785289    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785573    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785652    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785799    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785810    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785848    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785872    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.785904 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785904    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.785991    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.786004    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786056    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786088    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786195    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786233    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786249    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786292    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.786396 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786325    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.786755 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786638    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786755 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786691    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786755 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786700    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786755 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786726    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786760    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786769    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786785    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786802    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786833    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6mc\" (UniqueName: \"kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786866    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786851    2586 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786894    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.786898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786897    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786918    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786942    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlfl\" (UniqueName: \"kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786968    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787004    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787007    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdn8\" (UniqueName: \"kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787050    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787077    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787103    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787130    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787147    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787156    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787180    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787200    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787232    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787272    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.787374 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787312    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787342    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787349    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787383    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787386    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787427    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787440    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.787481    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.287453224 +0000 UTC m=+3.089448231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787503    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787530    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grgn\" (UniqueName: \"kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787558    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787581    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787587    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.786877    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787692    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787733    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787632    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.788196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787793    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787820    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787847    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787873    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787884    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787899    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787927    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787951    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.787993    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788017    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788066    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788078    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788085    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788129    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788134    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.788138    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788170    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.788946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788175    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.789750 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.788190    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.288175524 +0000 UTC m=+3.090170534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:29.789750 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.788621    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.789750 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.789371    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.789750 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.789494    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.791138 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.791096    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.791248 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.791230    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.792084 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.792016    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.798753 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.798735    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.810529 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.810501    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bkm9\" (UniqueName: \"kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.811360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.811301    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz42q\" (UniqueName: \"kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.811360 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.811325    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grgn\" (UniqueName: \"kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.811582 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.811561    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:29.811638 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.811588    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:29.811638 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.811610    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:29.812023 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:29.812001    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:30.311653518 +0000 UTC m=+3.113648528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:29.815628 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.814436    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdn8\" (UniqueName: \"kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.815628 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.814588    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6mc\" (UniqueName: \"kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:29.818506 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.818454    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g259q\" (UniqueName: \"kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.818609 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.818547    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvs8h\" (UniqueName: \"kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.819190 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.819129    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlfl\" (UniqueName: \"kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.820516 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.820494    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss25j\" (UniqueName: \"kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.909903 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.909868    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:12:29.919023 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.918997    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:12:29.923805 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.923780    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6466c095_0df6_4688_abc3_2f6f212471a0.slice/crio-a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3 WatchSource:0}: Error finding container a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3: Status 404 returned error can't find the container with id a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3
Oct 29 23:12:29.928497 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.928443    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8081555_b5de_4ffa_bbe7_e2931b269310.slice/crio-4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69 WatchSource:0}: Error finding container 4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69: Status 404 returned error can't find the container with id 4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69
Oct 29 23:12:29.941124 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.941101    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:12:29.941240 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.941219    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:29.943287 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.943269    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:12:29.945231 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.945205    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l9kgz"
Oct 29 23:12:29.963727 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.963703    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbe2362_88df_49bd_9692_e1311e28c396.slice/crio-9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a WatchSource:0}: Error finding container 9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a: Status 404 returned error can't find the container with id 9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a
Oct 29 23:12:29.965397 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.965372    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:12:29.968237 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.968212    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:12:29.971682 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:29.971663    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:29.976347 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.976321    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4fcb6f_b5d4_4806_882a_8145a71c40d3.slice/crio-f83e0b2871a149a852b736849518e287f1f9c3d1bc7f61b3890425966f493b6d WatchSource:0}: Error finding container f83e0b2871a149a852b736849518e287f1f9c3d1bc7f61b3890425966f493b6d: Status 404 returned error can't find the container with id f83e0b2871a149a852b736849518e287f1f9c3d1bc7f61b3890425966f493b6d
Oct 29 23:12:29.986446 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.986401    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5f497d2_8302_4707_b479_4dca872829e3.slice/crio-abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0 WatchSource:0}: Error finding container abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0: Status 404 returned error can't find the container with id abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0
Oct 29 23:12:29.987765 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:29.987744    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod867d79d0_81fd_4c82_9a3a_6d926b0a0ae4.slice/crio-763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc WatchSource:0}: Error finding container 763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc: Status 404 returned error can't find the container with id 763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc
Oct 29 23:12:30.291615 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.291583    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:30.291741 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.291662    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:30.291938 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.291910    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.292063 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.291967    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.29195278 +0000 UTC m=+4.093947772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:30.292395 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.292372    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.292483 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.292442    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.292425647 +0000 UTC m=+4.094420647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:30.392463 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.392386    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:30.392628 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.392522    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:30.392628 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.392536    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:30.392628 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.392544    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:30.392628 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:30.392587    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:31.392575246 +0000 UTC m=+4.194570239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:30.619043 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.618967    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:29 +0000 UTC" deadline="2028-07-19 22:07:52.274763432 +0000 UTC"
Oct 29 23:12:30.619043 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.619040    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15094h55m21.655726933s"
Oct 29 23:12:30.677062 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.676827    2586 generic.go:356] "Generic (PLEG): container finished" podID="8ad8bdceb419b718ac0187fcaf3ee427" containerID="69bc079f1819c4bf11be9d2e1b3d679b5c15b16d7f08c9ca1077f9353ce33f9c" exitCode=0
Oct 29 23:12:30.677062 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.677031    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerDied","Data":"69bc079f1819c4bf11be9d2e1b3d679b5c15b16d7f08c9ca1077f9353ce33f9c"}
Oct 29 23:12:30.686946 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.686797    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc"}
Oct 29 23:12:30.689528 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.689503    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4blvf" event={"ID":"e5f497d2-8302-4707-b479-4dca872829e3","Type":"ContainerStarted","Data":"abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0"}
Oct 29 23:12:30.698802 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.698779    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lhps4" event={"ID":"de4fcb6f-b5d4-4806-882a-8145a71c40d3","Type":"ContainerStarted","Data":"f83e0b2871a149a852b736849518e287f1f9c3d1bc7f61b3890425966f493b6d"}
Oct 29 23:12:30.715596 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.714119    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwtsk" event={"ID":"adbe2362-88df-49bd-9692-e1311e28c396","Type":"ContainerStarted","Data":"9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a"}
Oct 29 23:12:30.716481 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.716455    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk" event={"ID":"6466c095-0df6-4688-abc3-2f6f212471a0","Type":"ContainerStarted","Data":"a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3"}
Oct 29 23:12:30.743503 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.743474    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l9kgz" event={"ID":"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3","Type":"ContainerStarted","Data":"dde2fd0046a160a1605b25418199082b4ddfe15cc1df3aa54cbe0e4675ff2d71"}
Oct 29 23:12:30.751096 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.751030    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9h9g" event={"ID":"7e1cae4c-99a8-4eb9-a957-1b38f9460e55","Type":"ContainerStarted","Data":"7d7333f0931e99cf7615f2a785269d62f1853fb12dba98bddf6c82efc1cf6d6e"}
Oct 29 23:12:30.761463 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.761404    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerStarted","Data":"63d36568ecfae10c9e52634832bed8893c546ca332fcf6787353f60eab635a0b"}
Oct 29 23:12:30.763925 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.763854    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9m5t" event={"ID":"d8081555-b5de-4ffa-bbe7-e2931b269310","Type":"ContainerStarted","Data":"4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69"}
Oct 29 23:12:30.776915 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:30.776880    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal" event={"ID":"9bc940380ed911d9e6c421c48f4b1a6a","Type":"ContainerStarted","Data":"1ad9887bb3a6be48e60ec7dc3f7fe1c94cb638fe980145e381b9f25b28299cc9"}
Oct 29 23:12:31.299900 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.299599    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:31.300050 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.299934    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:31.300118 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.299835    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:31.300177 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.300133    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.300113383 +0000 UTC m=+6.102108580 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:31.300469 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.300065    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:31.300555 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.300512    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.300494057 +0000 UTC m=+6.102489053 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:31.400911 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.400877    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:31.401103 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.401090    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:31.401157 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.401109    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:31.401157 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.401123    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:31.401262 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.401176    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:33.401158344 +0000 UTC m=+6.203153343 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:31.619459 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.619397    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:29 +0000 UTC" deadline="2028-04-12 12:41:24.388267638 +0000 UTC"
Oct 29 23:12:31.619459 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.619451    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12733h28m52.768820531s"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.636171    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.636263    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.636620    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.636684    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.636855    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:31.636999 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:31.636922    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:31.793635 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.793528    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerStarted","Data":"0d632da8f4d6ccf2330cdf0f8ed709815ace5f8fe85b1384bba9ad6e0173e4ca"}
Oct 29 23:12:31.836305 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:31.836234    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal" podStartSLOduration=3.836213463 podStartE2EDuration="3.836213463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:30.819390053 +0000 UTC m=+3.621385085" watchObservedRunningTime="2026-10-29 23:12:31.836213463 +0000 UTC m=+4.638208478"
Oct 29 23:12:32.804181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:32.802802    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="76c4881882da6dd5d3670db0cd12c2ed850d874e46a8a7d08e8e347962d56c2d" exitCode=0
Oct 29 23:12:32.804181 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:32.803883    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"76c4881882da6dd5d3670db0cd12c2ed850d874e46a8a7d08e8e347962d56c2d"}
Oct 29 23:12:32.888132 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:32.887079    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" podStartSLOduration=4.887060952 podStartE2EDuration="4.887060952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:31.836530016 +0000 UTC m=+4.638525036" watchObservedRunningTime="2026-10-29 23:12:32.887060952 +0000 UTC m=+5.689055968"
Oct 29 23:12:33.320286 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.320203    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:33.320443 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.320308    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:33.320443 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.320434    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:33.320558 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.320499    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.320479999 +0000 UTC m=+10.122474999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:33.320619 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.320566    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:33.320619 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.320603    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.320591857 +0000 UTC m=+10.122586870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:33.421452 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.420841    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:33.421452 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.421038    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:33.421452 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.421059    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:33.421452 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.421071    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:33.421452 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.421124    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:37.4211083 +0000 UTC m=+10.223103307 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:33.637023 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.636990    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:33.637222 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.637003    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:33.637222 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.637118    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:33.637222 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.637196    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:33.637404 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:33.637249    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:33.637404 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:33.637324    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:34.816453 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:34.815676    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9h9g" event={"ID":"7e1cae4c-99a8-4eb9-a957-1b38f9460e55","Type":"ContainerStarted","Data":"e3db2a1252d3f0e5cd32bbf106f7c01ec278f623699d6e2b600aa20bd5914f9c"}
Oct 29 23:12:35.323209 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.323002    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:35.323704 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.323680    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:35.366357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.366313    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b9h9g" podStartSLOduration=4.610588487 podStartE2EDuration="8.366295375s" totalImagesPullingTime="3.755706888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:34.858488032 +0000 UTC m=+7.660483054" watchObservedRunningTime="2026-10-29 23:12:35.366295375 +0000 UTC m=+8.168290419"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.639076    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:35.639207    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.639287    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:35.639371    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.639720    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:35.640035 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:35.639807    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:35.823405 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.823346    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk" event={"ID":"6466c095-0df6-4688-abc3-2f6f212471a0","Type":"ContainerStarted","Data":"8f112fefa375eb313699fdf4418fac5ae463543090956e1b3e118163d31d5250"}
Oct 29 23:12:35.827087 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:35.826989    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g9m5t" event={"ID":"d8081555-b5de-4ffa-bbe7-e2931b269310","Type":"ContainerStarted","Data":"346390553efabca79fb3285357ac021885320773164fea2d5a3078f833a73501"}
Oct 29 23:12:36.828761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:36.828722    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:37.071326 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.071303    2586 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 29 23:12:37.353951 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.353904    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:37.353951 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.353962    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:37.355966 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.354113    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:37.355966 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.354160    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.354145759 +0000 UTC m=+18.156140751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:37.355966 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.354429    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:37.355966 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.354457    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.354448953 +0000 UTC m=+18.156443946 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:37.455560 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.454661    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:37.455560 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.454815    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:37.455560 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.454834    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:37.455560 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.454848    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:37.455560 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.454904    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.454885887 +0000 UTC m=+18.256880894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.638258    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.638382    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.638724    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.638797    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.640412    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:37.640534 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:37.640497    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:37.643652 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.643560    2586 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:12:37.071319881Z","UUID":"550f9416-de6b-4a32-8a32-d79374fc7776","Handler":null,"Name":"","Endpoint":""}
Oct 29 23:12:37.645733 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.645713    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 29 23:12:37.645919 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.645910    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 29 23:12:37.832359 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.832324    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4blvf" event={"ID":"e5f497d2-8302-4707-b479-4dca872829e3","Type":"ContainerStarted","Data":"dd7da93a7786515cd1df99e29e9e067582a410e1367389cdb6a839fab7b47c55"}
Oct 29 23:12:37.835589 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.835556    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk" event={"ID":"6466c095-0df6-4688-abc3-2f6f212471a0","Type":"ContainerStarted","Data":"352cda6c324ca0e328da1b5dfb715fc63fd4de082b6deed337705b4e7519b92b"}
Oct 29 23:12:37.837684 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.837245    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l9kgz" event={"ID":"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3","Type":"ContainerStarted","Data":"7a906e83d6fa8f9de0aacdb5bdd1c092f653428aabde776a3bcc049efd87fc44"}
Oct 29 23:12:37.840756 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.840201    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="7a5da1493cbdc3d4a14570059707f32239e8aa780da3b22b5afb89053c6e53b9" exitCode=0
Oct 29 23:12:37.840756 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.840235    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"7a5da1493cbdc3d4a14570059707f32239e8aa780da3b22b5afb89053c6e53b9"}
Oct 29 23:12:37.875777 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.875137    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g9m5t" podStartSLOduration=6.147329386 podStartE2EDuration="10.875119419s" totalImagesPullingTime="4.727790033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:35.869053427 +0000 UTC m=+8.671048463" watchObservedRunningTime="2026-10-29 23:12:37.875119419 +0000 UTC m=+10.677114435"
Oct 29 23:12:37.921452 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.921395    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-4blvf" podStartSLOduration=2.315663151 podStartE2EDuration="9.921377076s" totalImagesPullingTime="7.605713925s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:37.875624904 +0000 UTC m=+10.677619922" watchObservedRunningTime="2026-10-29 23:12:37.921377076 +0000 UTC m=+10.723372119"
Oct 29 23:12:37.965119 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:37.964382    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-l9kgz" podStartSLOduration=3.724846862 podStartE2EDuration="10.964370737s" totalImagesPullingTime="7.239523875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:37.964298118 +0000 UTC m=+10.766293135" watchObservedRunningTime="2026-10-29 23:12:37.964370737 +0000 UTC m=+10.766365751"
Oct 29 23:12:38.845598 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:38.845556    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk" event={"ID":"6466c095-0df6-4688-abc3-2f6f212471a0","Type":"ContainerStarted","Data":"71092b0bf4d8273d78026e64c8dbdc3506b3204fbf898edc0c66a376042c9103"}
Oct 29 23:12:38.848134 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:38.848104    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="5ea72a85f3a8e6a37b2f713d81864b82a38dad67e46cefdee41804624143b313" exitCode=0
Oct 29 23:12:38.848925 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:38.848898    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"5ea72a85f3a8e6a37b2f713d81864b82a38dad67e46cefdee41804624143b313"}
Oct 29 23:12:38.900025 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:38.899966    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk" podStartSLOduration=4.10058908 podStartE2EDuration="11.899953301s" totalImagesPullingTime="7.799364221s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:38.894505376 +0000 UTC m=+11.696500394" watchObservedRunningTime="2026-10-29 23:12:38.899953301 +0000 UTC m=+11.701948315"
Oct 29 23:12:39.636153 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.635927    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:39.636253 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.635989    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:39.636291 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:39.636251    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:39.636368 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:39.636338    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:39.636368 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.636030    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:39.636522 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:39.636455    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:39.852525 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.852487    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="d0441319ac506a6cb48c26b019de02a36e2e3d31d12f4ea959adf8d13445fc40" exitCode=0
Oct 29 23:12:39.852953 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.852584    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"d0441319ac506a6cb48c26b019de02a36e2e3d31d12f4ea959adf8d13445fc40"}
Oct 29 23:12:39.854689 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.854299    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lhps4" event={"ID":"de4fcb6f-b5d4-4806-882a-8145a71c40d3","Type":"ContainerStarted","Data":"44d7bdd5ecb2847d0d73e571155218bdea6321f492786b22bf70347329e5c6ed"}
Oct 29 23:12:39.855907 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.855884    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwtsk" event={"ID":"adbe2362-88df-49bd-9692-e1311e28c396","Type":"ContainerStarted","Data":"8fa76af2b2a33f982c73c6f79134d53af8f49b9264be525694e720092e7698ff"}
Oct 29 23:12:39.941108 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.941066    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nwtsk" podStartSLOduration=5.146955545 podStartE2EDuration="12.941053694s" totalImagesPullingTime="7.794098149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:39.940674225 +0000 UTC m=+12.742669239" watchObservedRunningTime="2026-10-29 23:12:39.941053694 +0000 UTC m=+12.743048708"
Oct 29 23:12:39.982544 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:39.982494    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lhps4" podStartSLOduration=2.379120493 podStartE2EDuration="11.982482141s" totalImagesPullingTime="9.603361648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:39.982226892 +0000 UTC m=+12.784221907" watchObservedRunningTime="2026-10-29 23:12:39.982482141 +0000 UTC m=+12.784477156"
Oct 29 23:12:41.636167 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:41.636136    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:41.636907 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:41.636249    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:41.636907 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:41.636555    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:41.636907 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:41.636629    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:41.636907 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:41.636753    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:41.636907 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:41.636807    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:41.863189 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:41.863155    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="b92fcf40dee55a08ba4ce58719dd49671bb35bd6ebd43f5697dbfa869fa65566" exitCode=0
Oct 29 23:12:41.863352 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:41.863201    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"b92fcf40dee55a08ba4ce58719dd49671bb35bd6ebd43f5697dbfa869fa65566"}
Oct 29 23:12:42.868878 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:42.868841    2586 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="80c2a155a024f0922b41624e391b8f1ac951e167b01fb04f781573a741b30e5a" exitCode=0
Oct 29 23:12:42.868878 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:42.868878    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerDied","Data":"80c2a155a024f0922b41624e391b8f1ac951e167b01fb04f781573a741b30e5a"}
Oct 29 23:12:43.637044 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:43.636838    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:43.637185 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:43.636869    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:43.637185 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:43.637117    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:43.637270 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:43.636907    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:43.637343 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:43.637325    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:43.637383 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:43.637243    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:43.874891 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:43.874860    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" event={"ID":"61f54971-c57c-405c-93a8-19fcc3e50908","Type":"ContainerStarted","Data":"3b33000d00cf3e12985b33ae5a36f99d980c08c368030f8a163554204bfe9c0d"}
Oct 29 23:12:43.951889 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:43.951849    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7g7tb" podStartSLOduration=7.170406347 podStartE2EDuration="16.951835833s" totalImagesPullingTime="9.781429486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:43.951249503 +0000 UTC m=+16.753244733" watchObservedRunningTime="2026-10-29 23:12:43.951835833 +0000 UTC m=+16.753830847"
Oct 29 23:12:44.882340 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882143    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"1a8e498dac289b9d3012f1be9c829133f394eecebb42f195a506e64831ec863a"}
Oct 29 23:12:44.882844 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882390    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"de175821f6bc343d21a60b4accc42b45e86df20b07ce64b680f29c3264b80413"}
Oct 29 23:12:44.882844 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882450    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"3fd2ade54c1b61377e572cc878a787f12f23e17d14d9d171eb4292c10fc0e188"}
Oct 29 23:12:44.882844 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882482    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"460d94ba40c4ad0d5ed73ca02e78c13aa1dba35327cbdd0251366b5d58e17585"}
Oct 29 23:12:44.882844 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882510    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"26b068dad3784c85a0bb5b709d1c4aa1c51b4427d1007342b0eb0770c272e346"}
Oct 29 23:12:44.882844 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:44.882543    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"563004243db10c689bd321f446d3ae02827ca33cc930006269f7050991d56198"}
Oct 29 23:12:45.416619 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.416590    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:45.416787 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.416630    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:45.416787 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.416743    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:45.416787 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.416771    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:45.416894 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.416793    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret podName:e29dccc7-5622-4583-b74e-220df6ff5a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.41677982 +0000 UTC m=+34.218774813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret") pod "global-pull-secret-syncer-64m88" (UID: "e29dccc7-5622-4583-b74e-220df6ff5a34") : object "kube-system"/"original-pull-secret" not registered
Oct 29 23:12:45.416894 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.416830    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.416815192 +0000 UTC m=+34.218810189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 29 23:12:45.517287 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.517260    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:45.517413 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.517372    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 29 23:12:45.517413 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.517387    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 29 23:12:45.517413 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.517396    2586 projected.go:196] Error preparing data for projected volume kube-api-access-zm68z for pod openshift-network-diagnostics/network-check-target-q9xkk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:45.517513 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.517435    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z podName:4fbaebde-1120-4288-a1bc-24d3f97bf85b nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.517425071 +0000 UTC m=+34.319420065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zm68z" (UniqueName: "kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z") pod "network-check-target-q9xkk" (UID: "4fbaebde-1120-4288-a1bc-24d3f97bf85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 29 23:12:45.636180 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.636154    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:45.636180 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.636173    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:45.636311 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.636274    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:45.636371 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.636346    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:45.636422 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:45.636398    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:45.636521 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:45.636499    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:46.888785 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:46.888751    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"cf9456594c08a80630fd5a105b0ecd746ae8c928ab83c166c2aa0a93ae9096f6"}
Oct 29 23:12:47.637143 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:47.637106    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:47.637339 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:47.637192    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:47.637339 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:47.637291    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:47.637455 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:47.637430    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:47.637495 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:47.637472    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:47.637561 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:47.637545    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:48.896938 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.896897    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" event={"ID":"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4","Type":"ContainerStarted","Data":"b803fd4de021956115c16809c856381b8ee79001243c71b219707fbecc129841"}
Oct 29 23:12:48.897847 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.897340    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:48.897847 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.897410    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:48.912424 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.912397    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:48.912662 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.912648    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:48.983644 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.983623    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:48.983739 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.983716    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:48.984143 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:48.984126    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:12:49.057584 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:49.057550    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8" podStartSLOduration=7.228543429 podStartE2EDuration="21.057539948s" totalImagesPullingTime="13.828996519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:49.05727203 +0000 UTC m=+21.859267056" watchObservedRunningTime="2026-10-29 23:12:49.057539948 +0000 UTC m=+21.859534962"
Oct 29 23:12:49.636753 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:49.636717    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:49.637116 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:49.636742    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:49.637245 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:49.637222    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:49.637292 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:49.637094    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:49.637292 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:49.636801    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:49.637367 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:49.637342    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:49.899743 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:49.899675    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:50.388851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.388825    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-64m88"]
Oct 29 23:12:50.389006 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.388923    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:50.389050 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:50.389024    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.393690    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqlgv"]
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.393788    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9xkk"]
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.393874    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:50.394006    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.394093    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:50.394689 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:50.394207    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:50.901873 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:50.901848    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:12:51.409516 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:51.409490    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:12:51.636226 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:51.636196    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:51.636226 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:51.636231    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:51.636484 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:51.636313    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:51.636484 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:51.636443    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:52.635851 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:52.635825    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:52.636207 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:52.635928    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:53.637043 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:53.636854    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:53.637529 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:53.636927    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:53.637529 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:53.637150    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:53.637529 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:53.637191    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:54.636029 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:54.636000    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:54.636168 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:54.636102    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:55.636880 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:55.636849    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:55.637249 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:55.637010    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-64m88" podUID="e29dccc7-5622-4583-b74e-220df6ff5a34"
Oct 29 23:12:55.637249 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:55.637054    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:55.637249 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:55.637141    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-q9xkk" podUID="4fbaebde-1120-4288-a1bc-24d3f97bf85b"
Oct 29 23:12:56.636142 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:56.636120    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:56.636281 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:56.636260    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlgv" podUID="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e"
Oct 29 23:12:56.956000 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:56.955929    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeReady"
Oct 29 23:12:56.956296 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:56.956046    2586 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 29 23:12:57.006328 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.006306    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhgrs"]
Oct 29 23:12:57.036164 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.036143    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-c68qr"]
Oct 29 23:12:57.036294 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.036274    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:12:57.047187 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.047168    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-88q8b"]
Oct 29 23:12:57.047290 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.047274    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:12:57.049480 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.049441    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-n4774"]
Oct 29 23:12:57.049590 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.049574    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:12:57.051866 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.051835    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-cddcfffcc-8wmcx"]
Oct 29 23:12:57.052668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.052650    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.055049 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.055025    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"]
Oct 29 23:12:57.055146 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.055131    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.061617 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.061419    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.062805 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.062671    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.076230 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076208    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 29 23:12:57.076353 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076332    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 29 23:12:57.076464 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076450    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 29 23:12:57.076537 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076498    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 29 23:12:57.076699 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076679    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gb2nj\""
Oct 29 23:12:57.076796 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.076741    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hfr96\""
Oct 29 23:12:57.081076 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.081056    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"]
Oct 29 23:12:57.081175 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.081124    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.093730 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.093712    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:12:57.093825 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.093811    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:12:57.096434 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.096418    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7v65t"]
Oct 29 23:12:57.096530 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.096517    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.100461 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.099682    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"]
Oct 29 23:12:57.100461 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.099815    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.104190 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.104172    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"]
Oct 29 23:12:57.104323 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.104301    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.107758 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.107717    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:12:57.108126 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.108109    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.112130 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.112114    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-n72kb"]
Oct 29 23:12:57.112252 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.112235    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.115767 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.115686    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"]
Oct 29 23:12:57.115869 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.115748    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.119228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.119210    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"]
Oct 29 23:12:57.119318 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.119303    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.122900 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.122881    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhgrs"]
Oct 29 23:12:57.123011 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.122913    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f8xtp"]
Oct 29 23:12:57.123011 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.122989    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:12:57.127041 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.127025    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hvb2t"]
Oct 29 23:12:57.127150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.127136    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.130255 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130236    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-c68qr"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130262    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-n4774"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130279    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cddcfffcc-8wmcx"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130298    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130317    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-88q8b"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130337    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8xtp"]
Oct 29 23:12:57.130345 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130345    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130361    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130380    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130398    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130410    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hvb2t"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130423    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130465    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130497    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130517    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7v65t"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130536    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"]
Oct 29 23:12:57.130642 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.130555    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-n72kb"]
Oct 29 23:12:57.161951 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.161928    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 29 23:12:57.162234 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162199    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.162234 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162212    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.162349 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162246    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.162443 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162368    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.162443 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162381    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 29 23:12:57.162443 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162375    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.162443 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162416    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 29 23:12:57.162806 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162784    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 29 23:12:57.162898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162829    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.162898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162866    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 29 23:12:57.163021 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162929    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 29 23:12:57.163021 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162941    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163021 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.162944    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163186 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163024    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163186 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163058    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163186 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163109    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163191    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163227    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163239    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163261    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163267    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163273    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163287    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163298    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163324    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163324    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 29 23:12:57.163335 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163338    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 29 23:12:57.163955 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.163406    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 29 23:12:57.178677 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.178657    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 29 23:12:57.191588 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191565    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 29 23:12:57.191588 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191571    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 29 23:12:57.191726 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191605    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 29 23:12:57.191726 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191671    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 29 23:12:57.191726 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191706    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l8rj6\""
Oct 29 23:12:57.191726 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191720    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191725    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191713    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191756    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gj5kk\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191765    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qprkx\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191790    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191801    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191832    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191835    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pphcq\""
Oct 29 23:12:57.191895 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191842    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.191937    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-g2zrz\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192034    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2dl5f\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192093    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tldxj\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192149    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2p5c8\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192191    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192238    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h9zs2\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192244    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-6nt2h\""
Oct 29 23:12:57.192264 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.192254    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 29 23:12:57.204663 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204639    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.204752 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204669    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.204752 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204692    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.204752 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204719    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2b6r\" (UniqueName: \"kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.204849 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204768    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.204882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204857    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.204938 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204924    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whdld\" (UniqueName: \"kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld\") pod \"kas-connection-checker-685895c7c9-hhgrs\" (UID: \"7bd53292-a0c6-41a8-8a39-41a27853cb73\") " pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:12:57.204970 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.204945    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.205021 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205007    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.205054 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205038    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.205088 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205070    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.205119 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205090    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfnd\" (UniqueName: \"kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd\") pod \"kas-connection-checker-685895c7c9-c68qr\" (UID: \"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7\") " pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:12:57.205119 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205106    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rpww\" (UniqueName: \"kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.205171 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205129    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.205171 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205150    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.205171 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205167    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.205275 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205181    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.205275 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205203    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205275 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205257    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dnjr\" (UniqueName: \"kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.205370 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205282    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.205370 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205300    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205370 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205350    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.205474 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205381    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205474 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205402    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205474 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205442    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205581 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205485    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.205581 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205510    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205581 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205533    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.205581 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205562    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.205724 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205627    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.205724 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205659    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k4nt\" (UniqueName: \"kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt\") pod \"volume-data-source-validator-58c9f5f46d-g2czn\" (UID: \"f5c0147f-93d0-431f-a294-93c26838eff3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:12:57.205724 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205696    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.205724 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205714    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205729    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205746    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205762    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205790    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205816    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.205852 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205831    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205878    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2db6t\" (UniqueName: \"kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205902    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205917    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205954    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjv9k\" (UniqueName: \"kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.205996    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206031    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mn8r\" (UniqueName: \"kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r\") pod \"kas-connection-checker-685895c7c9-88q8b\" (UID: \"86fa6768-086e-4434-9d30-cc860a28acc6\") " pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:12:57.206086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206057    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.206357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206114    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.206357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206132    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxwxd\" (UniqueName: \"kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.206357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206167    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl9zc\" (UniqueName: \"kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.206357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.206221    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwp8j\" (UniqueName: \"kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j\") pod \"network-check-source-6f78d59f99-6djdh\" (UID: \"0641c627-1da8-47bc-bca1-04ade039e493\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:12:57.307448 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307427    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.307543 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307459    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.307543 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307489    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.307543 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307525    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whdld\" (UniqueName: \"kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld\") pod \"kas-connection-checker-685895c7c9-hhgrs\" (UID: \"7bd53292-a0c6-41a8-8a39-41a27853cb73\") " pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:12:57.307636 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307559    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.307636 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.307623    2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:57.307636 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.307631    2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307643    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.307678    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.807660341 +0000 UTC m=+30.609655349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.307695    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.807686405 +0000 UTC m=+30.609681401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307711    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307739    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.307771 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307761    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfnd\" (UniqueName: \"kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd\") pod \"kas-connection-checker-685895c7c9-c68qr\" (UID: \"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7\") " pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:12:57.308053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307782    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpww\" (UniqueName: \"kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.308053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307808    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqllh\" (UniqueName: \"kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.308053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307835    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.308053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307860    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.308053 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.307885    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.308271 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308250    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.308327 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308300    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.308382 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.308334    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.808314073 +0000 UTC m=+30.610309082 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:12:57.308382 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308361    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308487 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308408    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnjr\" (UniqueName: \"kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.308487 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308434    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.308487 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308458    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308487 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.308476    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.308491    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308490    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308523    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.308531    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.808516339 +0000 UTC m=+30.610511351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308590    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308621    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308668 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308650    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308687    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308706    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308721    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308744    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308762    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308787    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308802    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4nt\" (UniqueName: \"kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt\") pod \"volume-data-source-validator-58c9f5f46d-g2czn\" (UID: \"f5c0147f-93d0-431f-a294-93c26838eff3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308825    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308843    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308857    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308872    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.308995 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.308894    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.309464 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.309300    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 29 23:12:57.309464 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.309347    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls podName:37f1ad0a-20e9-45bb-9340-37f5dc7410d3 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.809332101 +0000 UTC m=+30.611327111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q47dp" (UID: "37f1ad0a-20e9-45bb-9340-37f5dc7410d3") : secret "samples-operator-tls" not found
Oct 29 23:12:57.309464 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309437    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.309597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309472    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.309597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309500    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.309597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309533    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2db6t\" (UniqueName: \"kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.309597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309562    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309618    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309650    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjv9k\" (UniqueName: \"kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.309671    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309676    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309713    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mn8r\" (UniqueName: \"kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r\") pod \"kas-connection-checker-685895c7c9-88q8b\" (UID: \"86fa6768-086e-4434-9d30-cc860a28acc6\") " pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:12:57.309761 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309743    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.310055 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309781    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.310055 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309810    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxwxd\" (UniqueName: \"kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.310055 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.309841    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89dfq\" (UniqueName: \"kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.310055 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310020    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310057    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9zc\" (UniqueName: \"kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310104    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwp8j\" (UniqueName: \"kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j\") pod \"network-check-source-6f78d59f99-6djdh\" (UID: \"0641c627-1da8-47bc-bca1-04ade039e493\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310155    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310184    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310208    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.309684    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:12:57.310241 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.310236    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2b6r\" (UniqueName: \"kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.310550 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.310283    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.810267908 +0000 UTC m=+30.612262925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:12:57.310550 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.310169    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:12:57.310550 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.310325    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.810315052 +0000 UTC m=+30.612310080 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:12:57.325162 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.324494    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.325162 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.324506    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.325162 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.324937    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.326882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.325812    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.326882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.326054    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.326882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.326210    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.326882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.326701    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.326882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.326789    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.327935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.327371    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.327935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.327767    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.328061 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.327944    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.328116 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328091    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328158    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328252    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328292    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328647    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328730    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.328761    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.329148    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.329301    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.329352    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.329993    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.330026    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.330985 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.330228    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.332444 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.332415    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpww\" (UniqueName: \"kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.338841 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.338807    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2b6r\" (UniqueName: \"kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.338921 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.338904    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4nt\" (UniqueName: \"kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt\") pod \"volume-data-source-validator-58c9f5f46d-g2czn\" (UID: \"f5c0147f-93d0-431f-a294-93c26838eff3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.338904    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjv9k\" (UniqueName: \"kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.339288    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2db6t\" (UniqueName: \"kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.339522    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9zc\" (UniqueName: \"kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.339558    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.339617    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.339852    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxwxd\" (UniqueName: \"kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340034    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfnd\" (UniqueName: \"kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd\") pod \"kas-connection-checker-685895c7c9-c68qr\" (UID: \"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7\") " pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:12:57.340090 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340037    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.340520 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340099    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mn8r\" (UniqueName: \"kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r\") pod \"kas-connection-checker-685895c7c9-88q8b\" (UID: \"86fa6768-086e-4434-9d30-cc860a28acc6\") " pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:12:57.340520 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340122    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnjr\" (UniqueName: \"kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.340811 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340791    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwp8j\" (UniqueName: \"kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j\") pod \"network-check-source-6f78d59f99-6djdh\" (UID: \"0641c627-1da8-47bc-bca1-04ade039e493\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:12:57.340948 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340924    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whdld\" (UniqueName: \"kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld\") pod \"kas-connection-checker-685895c7c9-hhgrs\" (UID: \"7bd53292-a0c6-41a8-8a39-41a27853cb73\") " pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:12:57.341086 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.340965    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.341156 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.341100    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.341399 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.341380    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.349229 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.349212    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:12:57.372710 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.372691    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:12:57.389790 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.389773    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:12:57.395966 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.395946    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:12:57.406229 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.406176    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411779    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411825    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqllh\" (UniqueName: \"kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411867    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411893    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411947    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89dfq\" (UniqueName: \"kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.411962    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.412220    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.412301 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.412298    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 29 23:12:57.412682 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.412338    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls podName:d42a6856-295d-4cf1-8a6e-691fcc073a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.912322904 +0000 UTC m=+30.714317910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls") pod "dns-default-f8xtp" (UID: "d42a6856-295d-4cf1-8a6e-691fcc073a34") : secret "dns-default-metrics-tls" not found
Oct 29 23:12:57.412862 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.412845    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.417701 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.417667    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.423810 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.423746    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:12:57.430448 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.430373    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:12:57.435573 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.435527    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:12:57.436961 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.436883    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89dfq\" (UniqueName: \"kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.437530 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.437512    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqllh\" (UniqueName: \"kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.464581 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.464547    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:12:57.472208 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.472165    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:12:57.637537 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.637511    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:12:57.638150 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.637733    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:12:57.654491 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.654460    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 29 23:12:57.654598 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.654546    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-phkn8\""
Oct 29 23:12:57.654840 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.654657    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-89z4k\""
Oct 29 23:12:57.695434 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.695412    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd53292_a0c6_41a8_8a39_41a27853cb73.slice/crio-ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d WatchSource:0}: Error finding container ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d: Status 404 returned error can't find the container with id ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d
Oct 29 23:12:57.697295 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.697278    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-hhgrs"]
Oct 29 23:12:57.699467 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.699449    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-88q8b"]
Oct 29 23:12:57.700497 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.700478    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fa6768_086e_4434_9d30_cc860a28acc6.slice/crio-7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe WatchSource:0}: Error finding container 7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe: Status 404 returned error can't find the container with id 7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe
Oct 29 23:12:57.703936 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.703911    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-c68qr"]
Oct 29 23:12:57.752283 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.752257    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-n4774"]
Oct 29 23:12:57.762791 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.762764    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"]
Oct 29 23:12:57.766687 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.766657    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61d9ca2_dabd_4378_bee7_ecacbfd366a6.slice/crio-6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740 WatchSource:0}: Error finding container 6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740: Status 404 returned error can't find the container with id 6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740
Oct 29 23:12:57.770091 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.770067    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5c0147f_93d0_431f_a294_93c26838eff3.slice/crio-2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099 WatchSource:0}: Error finding container 2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099: Status 404 returned error can't find the container with id 2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099
Oct 29 23:12:57.794752 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.794675    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-7v65t"]
Oct 29 23:12:57.800742 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.799766    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc765f10e_91f1_4c9f_9ffd_2740b2bde8b9.slice/crio-37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06 WatchSource:0}: Error finding container 37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06: Status 404 returned error can't find the container with id 37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06
Oct 29 23:12:57.815748 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.815723    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:57.815849 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.815795    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:57.815849 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.815831    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.815958 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.815869    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:57.815958 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.815924    2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:57.816071 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.815988    2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:12:57.816071 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816007    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.815968227 +0000 UTC m=+31.617963246 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:57.816071 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.815932    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:57.816071 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816028    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 29 23:12:57.816071 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816046    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.816029387 +0000 UTC m=+31.618024397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816093    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls podName:37f1ad0a-20e9-45bb-9340-37f5dc7410d3 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.816059652 +0000 UTC m=+31.618054664 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q47dp" (UID: "37f1ad0a-20e9-45bb-9340-37f5dc7410d3") : secret "samples-operator-tls" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816114    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.816105112 +0000 UTC m=+31.618100104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816139    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.816141    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816152    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.816171    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816187    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.81617531 +0000 UTC m=+31.618170303 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816230    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816267    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.816260679 +0000 UTC m=+31.618255713 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:12:57.816324 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816321    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:57.816794 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816333    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:12:57.816794 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.816368    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.81635622 +0000 UTC m=+31.618351235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:12:57.845073 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.844713    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"]
Oct 29 23:12:57.846604 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.846576    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe93ab9c_ae4e_48af_a423_74e32a601101.slice/crio-055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d WatchSource:0}: Error finding container 055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d: Status 404 returned error can't find the container with id 055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d
Oct 29 23:12:57.847390 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.847359    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"]
Oct 29 23:12:57.847865 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:12:57.847834    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode525b897_14cf_4e83_a016_60ab539f490d.slice/crio-5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466 WatchSource:0}: Error finding container 5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466: Status 404 returned error can't find the container with id 5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466
Oct 29 23:12:57.848521 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.848501    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"]
Oct 29 23:12:57.867938 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.867919    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hvb2t"]
Oct 29 23:12:57.916999 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.916963    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:57.917101 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.917077    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 29 23:12:57.917165 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:57.917124    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls podName:d42a6856-295d-4cf1-8a6e-691fcc073a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:58.917106962 +0000 UTC m=+31.719101969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls") pod "dns-default-f8xtp" (UID: "d42a6856-295d-4cf1-8a6e-691fcc073a34") : secret "dns-default-metrics-tls" not found
Oct 29 23:12:57.917759 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.917735    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hvb2t" event={"ID":"f63a0452-9661-4fbb-9396-9ac7b5524dd3","Type":"ContainerStarted","Data":"35489098b1a4431ee682829332159e879813364e8b0050e43b26fd8151ebc2af"}
Oct 29 23:12:57.918786 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.918760    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-n4774" event={"ID":"e61d9ca2-dabd-4378-bee7-ecacbfd366a6","Type":"ContainerStarted","Data":"6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740"}
Oct 29 23:12:57.919779 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.919756    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8" event={"ID":"e525b897-14cf-4e83-a016-60ab539f490d","Type":"ContainerStarted","Data":"5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466"}
Oct 29 23:12:57.921109 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.921086    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-88q8b" event={"ID":"86fa6768-086e-4434-9d30-cc860a28acc6","Type":"ContainerStarted","Data":"ebc83d9a1b03704045438565933fdf72b61b4078d509491e73f527ad93826559"}
Oct 29 23:12:57.921180 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.921124    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-88q8b" event={"ID":"86fa6768-086e-4434-9d30-cc860a28acc6","Type":"ContainerStarted","Data":"7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe"}
Oct 29 23:12:57.922221 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.922204    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh" event={"ID":"0641c627-1da8-47bc-bca1-04ade039e493","Type":"ContainerStarted","Data":"90378bcab5f1bb1f2b080b3f0d98d0f5bd275f77c3d953da15dfb8398e2dc8ea"}
Oct 29 23:12:57.923228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.923211    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" event={"ID":"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9","Type":"ContainerStarted","Data":"37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06"}
Oct 29 23:12:57.924285 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.924268    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst" event={"ID":"fe93ab9c-ae4e-48af-a423-74e32a601101","Type":"ContainerStarted","Data":"055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d"}
Oct 29 23:12:57.925351 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.925328    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn" event={"ID":"f5c0147f-93d0-431f-a294-93c26838eff3","Type":"ContainerStarted","Data":"2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099"}
Oct 29 23:12:57.926663 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.926640    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-c68qr" event={"ID":"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7","Type":"ContainerStarted","Data":"3dcb742859c068c0868081816005ef14f3a37e5a8d4d059d40771c74cd16cb7f"}
Oct 29 23:12:57.926759 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.926680    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-c68qr" event={"ID":"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7","Type":"ContainerStarted","Data":"994759f998f6643cbc441a452fb4e78c956bbd1da4a05c8221f4023dfe8a8907"}
Oct 29 23:12:57.927941 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.927923    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-hhgrs" event={"ID":"7bd53292-a0c6-41a8-8a39-41a27853cb73","Type":"ContainerStarted","Data":"ae6b367aa883ff85f418b83130f1017db7fa37c62656ade4a4ba41218570c690"}
Oct 29 23:12:57.928037 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.927956    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-hhgrs" event={"ID":"7bd53292-a0c6-41a8-8a39-41a27853cb73","Type":"ContainerStarted","Data":"ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d"}
Oct 29 23:12:57.965952 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:57.965890    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-88q8b" podStartSLOduration=1952.965878695 podStartE2EDuration="32m32.965878695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:57.965039458 +0000 UTC m=+30.767034473" watchObservedRunningTime="2026-10-29 23:12:57.965878695 +0000 UTC m=+30.767873709"
Oct 29 23:12:58.011021 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.010959    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-hhgrs" podStartSLOduration=1953.010946561 podStartE2EDuration="32m33.010946561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:58.010685828 +0000 UTC m=+30.812680825" watchObservedRunningTime="2026-10-29 23:12:58.010946561 +0000 UTC m=+30.812941577"
Oct 29 23:12:58.062169 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.062138    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-c68qr" podStartSLOduration=1953.062129912 podStartE2EDuration="32m33.062129912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:58.061252749 +0000 UTC m=+30.863247763" watchObservedRunningTime="2026-10-29 23:12:58.062129912 +0000 UTC m=+30.864124956"
Oct 29 23:12:58.636842 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.635984    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:12:58.653196 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.653148    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 29 23:12:58.653562 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.653525    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vcdm\""
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826147    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826200    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826231    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826280    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826316    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826357    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls podName:37f1ad0a-20e9-45bb-9340-37f5dc7410d3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.826337815 +0000 UTC m=+33.628332826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q47dp" (UID: "37f1ad0a-20e9-45bb-9340-37f5dc7410d3") : secret "samples-operator-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826412    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826428    2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826451    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826479    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.82646371 +0000 UTC m=+33.628458703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.826509    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826529    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.826519288 +0000 UTC m=+33.628514299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826574    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826585    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826620    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.826608398 +0000 UTC m=+33.628603390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:12:58.827028 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826678    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:12:58.827901 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826689    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:12:58.827901 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826718    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.826709173 +0000 UTC m=+33.628704185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:12:58.827901 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826778    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:12:58.827901 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.826812    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.826801739 +0000 UTC m=+33.628796734 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:12:58.828233 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.828151    2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:12:58.828233 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.828211    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.828196252 +0000 UTC m=+33.630191263 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:12:58.928245 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:58.927823    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:12:58.928245 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.928010    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 29 23:12:58.928245 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:12:58.928069    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls podName:d42a6856-295d-4cf1-8a6e-691fcc073a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:00.928050988 +0000 UTC m=+33.730045987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls") pod "dns-default-f8xtp" (UID: "d42a6856-295d-4cf1-8a6e-691fcc073a34") : secret "dns-default-metrics-tls" not found
Oct 29 23:12:59.908267 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:59.908181    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:12:59.942043 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:59.942004    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn" event={"ID":"f5c0147f-93d0-431f-a294-93c26838eff3","Type":"ContainerStarted","Data":"1126af69206d3a431e812cec5ecdaac969939db790a4b468c902b7daec54e5ca"}
Oct 29 23:12:59.987497 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:12:59.987415    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn" podStartSLOduration=1919.148666111 podStartE2EDuration="32m0.98739587s" totalImagesPullingTime="1.838729759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:59.9859594 +0000 UTC m=+32.787954411" watchObservedRunningTime="2026-10-29 23:12:59.98739587 +0000 UTC m=+32.789390886"
Oct 29 23:13:00.113015 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.112960    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:00.319382 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.319240    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850096    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850170    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850207    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850234    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850259    2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850298    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850325    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850305738 +0000 UTC m=+37.652300748 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850357    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850366    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.850388    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850401    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls podName:37f1ad0a-20e9-45bb-9340-37f5dc7410d3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850390178 +0000 UTC m=+37.652385186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q47dp" (UID: "37f1ad0a-20e9-45bb-9340-37f5dc7410d3") : secret "samples-operator-tls" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850460    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850491    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850480516 +0000 UTC m=+37.652475523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850513    2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:00.850815 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850542    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850533474 +0000 UTC m=+37.652528469 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850602    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850593719 +0000 UTC m=+37.652588730 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850664    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850674    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850701    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850692152 +0000 UTC m=+37.652687158 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850748    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850757    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:13:00.851804 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.850781    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.850774099 +0000 UTC m=+37.652769107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:13:00.951014 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:00.950960    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:00.951954 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.951933    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 29 23:13:00.952097 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:00.952002    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls podName:d42a6856-295d-4cf1-8a6e-691fcc073a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.9519841 +0000 UTC m=+37.753979109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls") pod "dns-default-f8xtp" (UID: "d42a6856-295d-4cf1-8a6e-691fcc073a34") : secret "dns-default-metrics-tls" not found
Oct 29 23:13:01.284475 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.284350    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:01.456394 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.456368    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:01.456518 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.456495    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:01.456633 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:01.456618    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 29 23:13:01.456698 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:01.456678    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:13:33.456659029 +0000 UTC m=+66.258654039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : secret "metrics-daemon-secret" not found
Oct 29 23:13:01.458734 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.458710    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:01.476821 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.476796    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:01.565673 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.565590    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:01.572002 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.571961    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:01.572960 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.572939    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:01.578586 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.578561    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:01.898704 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.898675    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-64m88"]
Oct 29 23:13:01.901629 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:01.901597    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29dccc7_5622_4583_b74e_220df6ff5a34.slice/crio-d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a WatchSource:0}: Error finding container d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a: Status 404 returned error can't find the container with id d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a
Oct 29 23:13:01.934012 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.933962    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-q9xkk"]
Oct 29 23:13:01.948319 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:01.948287    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbaebde_1120_4288_a1bc_24d3f97bf85b.slice/crio-2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379 WatchSource:0}: Error finding container 2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379: Status 404 returned error can't find the container with id 2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379
Oct 29 23:13:01.950845 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.950813    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-64m88" event={"ID":"e29dccc7-5622-4583-b74e-220df6ff5a34","Type":"ContainerStarted","Data":"d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a"}
Oct 29 23:13:01.953111 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.953093    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/0.log"
Oct 29 23:13:01.953433 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.953128    2586 generic.go:356] "Generic (PLEG): container finished" podID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" containerID="d8531f7b3e0a5a0f864dcd0ddf82120443c2e2dd21ece98d0ef12dc87ff09c5c" exitCode=255
Oct 29 23:13:01.953433 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.953160    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" event={"ID":"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9","Type":"ContainerDied","Data":"d8531f7b3e0a5a0f864dcd0ddf82120443c2e2dd21ece98d0ef12dc87ff09c5c"}
Oct 29 23:13:01.953433 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:01.953427    2586 scope.go:122] "RemoveContainer" containerID="d8531f7b3e0a5a0f864dcd0ddf82120443c2e2dd21ece98d0ef12dc87ff09c5c"
Oct 29 23:13:02.961378 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.961346    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:02.962811 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.962783    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/0.log"
Oct 29 23:13:02.962941 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.962831    2586 generic.go:356] "Generic (PLEG): container finished" podID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2" exitCode=255
Oct 29 23:13:02.963357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.963065    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" event={"ID":"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9","Type":"ContainerDied","Data":"2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2"}
Oct 29 23:13:02.963357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.963129    2586 scope.go:122] "RemoveContainer" containerID="d8531f7b3e0a5a0f864dcd0ddf82120443c2e2dd21ece98d0ef12dc87ff09c5c"
Oct 29 23:13:02.963357 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.963243    2586 scope.go:122] "RemoveContainer" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2"
Oct 29 23:13:02.964067 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:02.963457    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7v65t_openshift-console-operator(c765f10e-91f1-4c9f-9ffd-2740b2bde8b9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" podUID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9"
Oct 29 23:13:02.967692 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.967668    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst" event={"ID":"fe93ab9c-ae4e-48af-a423-74e32a601101","Type":"ContainerStarted","Data":"d2edeb105d387f401aa36ffab4f29d9243d2f142340925bf6ad6e194ff535f8f"}
Oct 29 23:13:02.970066 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.970044    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-n4774" event={"ID":"e61d9ca2-dabd-4378-bee7-ecacbfd366a6","Type":"ContainerStarted","Data":"1b53185c777da105ce927a63497334004ee1c15db29729f98555da099a9e5feb"}
Oct 29 23:13:02.971824 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.971796    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8" event={"ID":"e525b897-14cf-4e83-a016-60ab539f490d","Type":"ContainerStarted","Data":"b9e57c69414e3ed4408d6e7ffd55541e0934e13237bda6d6c5a4e764c21d541e"}
Oct 29 23:13:02.973293 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:02.973235    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9xkk" event={"ID":"4fbaebde-1120-4288-a1bc-24d3f97bf85b","Type":"ContainerStarted","Data":"2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379"}
Oct 29 23:13:03.053597 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.053172    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-n4774" podStartSLOduration=1920.268646208 podStartE2EDuration="32m5.053159607s" totalImagesPullingTime="4.784513399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:03.052890678 +0000 UTC m=+35.854885697" watchObservedRunningTime="2026-10-29 23:13:03.053159607 +0000 UTC m=+35.855154622"
Oct 29 23:13:03.105758 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.105711    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst" podStartSLOduration=1919.400202401 podStartE2EDuration="32m4.105692891s" totalImagesPullingTime="4.70549049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:03.105516327 +0000 UTC m=+35.907511345" watchObservedRunningTime="2026-10-29 23:13:03.105692891 +0000 UTC m=+35.907687908"
Oct 29 23:13:03.147020 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.146680    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8" podStartSLOduration=1919.426034939 podStartE2EDuration="32m4.146662196s" totalImagesPullingTime="4.720627257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:03.146165855 +0000 UTC m=+35.948160871" watchObservedRunningTime="2026-10-29 23:13:03.146662196 +0000 UTC m=+35.948657213"
Oct 29 23:13:03.528506 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.528470    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:03.680896 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.680408    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:03.737072 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.737038    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"]
Oct 29 23:13:03.766720 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.760039    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"]
Oct 29 23:13:03.766720 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.760534    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:03.780347 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.776122    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:03.785035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.781419    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 29 23:13:03.785035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.781538    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ss48m\""
Oct 29 23:13:03.785035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.781750    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:03.785035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.784210    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"]
Oct 29 23:13:03.785035 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.784255    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"]
Oct 29 23:13:03.789442 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.789418    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsbhp\" (UniqueName: \"kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp\") pod \"migrator-66df5d496b-sz7lb\" (UID: \"e82f1b60-106c-4927-ad20-18a9cdf92709\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:03.789541 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.789526    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x2bw\" (UniqueName: \"kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw\") pod \"migrator-66df5d496b-9ft4t\" (UID: \"e0347d66-da9b-4703-85f2-cfe2f25eb1d9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:03.827105 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.827079    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:03.890776 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.890746    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x2bw\" (UniqueName: \"kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw\") pod \"migrator-66df5d496b-9ft4t\" (UID: \"e0347d66-da9b-4703-85f2-cfe2f25eb1d9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:03.890908 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.890878    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbhp\" (UniqueName: \"kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp\") pod \"migrator-66df5d496b-sz7lb\" (UID: \"e82f1b60-106c-4927-ad20-18a9cdf92709\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:03.913225 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.913200    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbhp\" (UniqueName: \"kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp\") pod \"migrator-66df5d496b-sz7lb\" (UID: \"e82f1b60-106c-4927-ad20-18a9cdf92709\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:03.914677 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.914654    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x2bw\" (UniqueName: \"kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw\") pod \"migrator-66df5d496b-9ft4t\" (UID: \"e0347d66-da9b-4703-85f2-cfe2f25eb1d9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:03.940681 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.940621    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-jhgs7"]
Oct 29 23:13:03.977023 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.968143    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-jhgs7"]
Oct 29 23:13:03.977023 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.968271    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:03.980353 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.980324    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh" event={"ID":"0641c627-1da8-47bc-bca1-04ade039e493","Type":"ContainerStarted","Data":"42e2c5ca45acd5527255b98d8e17c544f82ffd5ab274e30a9dd4a9e286768e93"}
Oct 29 23:13:03.985341 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.984771    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 29 23:13:03.985341 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.985162    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hn6wx\""
Oct 29 23:13:03.985341 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.985216    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 29 23:13:03.985341 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.985253    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 29 23:13:03.985341 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.985305    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 29 23:13:03.985861 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.985791    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:03.991790 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.991766    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:03.991900 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.991837    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75xdt\" (UniqueName: \"kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:03.991900 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.991890    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:03.993594 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.993475    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:03.993935 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.993914    2586 scope.go:122] "RemoveContainer" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2"
Oct 29 23:13:03.994141 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:03.994116    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7v65t_openshift-console-operator(c765f10e-91f1-4c9f-9ffd-2740b2bde8b9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" podUID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9"
Oct 29 23:13:03.996463 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:03.996045    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hvb2t" event={"ID":"f63a0452-9661-4fbb-9396-9ac7b5524dd3","Type":"ContainerStarted","Data":"e3a2ba582f496c1ea953987264dce662e96fe7774a13c6a90413d7cfa573ad43"}
Oct 29 23:13:04.055004 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.037621    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mcxnv"]
Oct 29 23:13:04.061048 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.059081    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.061048 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.060957    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mcxnv"]
Oct 29 23:13:04.081183 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.081163    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:04.085162 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.085142    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 29 23:13:04.085489 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.085420    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 29 23:13:04.085489 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.085459    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sx9m6\""
Oct 29 23:13:04.092898 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.092873    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.093094 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093074    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.093190 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093176    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75xdt\" (UniqueName: \"kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.093265 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093249    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.093320 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093296    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.093379 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093327    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvxnl\" (UniqueName: \"kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.093461 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093439    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.093568 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.093478    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.097784 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.097763    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.098405 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.098381    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.099259 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.099239    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.100030 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.099797    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh" podStartSLOduration=1919.536573621 podStartE2EDuration="32m5.099782421s" totalImagesPullingTime="5.5632088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:04.097486076 +0000 UTC m=+36.899481116" watchObservedRunningTime="2026-10-29 23:13:04.099782421 +0000 UTC m=+36.901777437"
Oct 29 23:13:04.103588 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.103523    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:04.104948 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.104797    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:04.121039 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.120994    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75xdt\" (UniqueName: \"kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.123558 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.123537    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:04.143408 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.143384    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:04.144245 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.144199    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hvb2t" podStartSLOduration=2.001457912 podStartE2EDuration="7.144182723s" totalImagesPullingTime="5.142724811s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:04.142352941 +0000 UTC m=+36.944347955" watchObservedRunningTime="2026-10-29 23:13:04.144182723 +0000 UTC m=+36.946177742"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198349    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198395    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198460    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198510    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198526    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxnl\" (UniqueName: \"kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198546    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198595    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198625    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.198647    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.199228 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.199067    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.200336 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.200012    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.200336 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.200309    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:04.200703 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.200672    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.700635348 +0000 UTC m=+37.502630357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:04.200808 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.200754    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:04.200900 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.200819    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:04.700804716 +0000 UTC m=+37.502799710 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:04.223458 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.223413    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxnl\" (UniqueName: \"kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.293867 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.293655    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:04.299599 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.299568    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.299728 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.299675    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.299799 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.299743    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.299853 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.299831    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.300320 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.300279    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.300936 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.300908    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.301522 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.301451    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.301919 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.301821    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:04.328177 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.328141    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.440286 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.440249    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:04.447999 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.447958    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"]
Oct 29 23:13:04.449293 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:04.449044    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0347d66_da9b_4703_85f2_cfe2f25eb1d9.slice/crio-46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35 WatchSource:0}: Error finding container 46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35: Status 404 returned error can't find the container with id 46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35
Oct 29 23:13:04.457404 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.457385    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"]
Oct 29 23:13:04.459867 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.459843    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:04.460675 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:04.460647    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82f1b60_106c_4927_ad20_18a9cdf92709.slice/crio-3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad WatchSource:0}: Error finding container 3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad: Status 404 returned error can't find the container with id 3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad
Oct 29 23:13:04.632251 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.632220    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-jhgs7"]
Oct 29 23:13:04.633216 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:04.633192    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1839b99_bbb7_4a20_9507_7d72dbbe354d.slice/crio-1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0 WatchSource:0}: Error finding container 1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0: Status 404 returned error can't find the container with id 1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0
Oct 29 23:13:04.703765 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.703698    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.703864 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.703763    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:04.703933 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.703917    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:04.704011 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.703999    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:05.703960775 +0000 UTC m=+38.505955773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:04.704350 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.704334    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:04.704413 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.704382    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:05.704371629 +0000 UTC m=+38.506366642 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:04.766783 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.766644    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:04.770209 ip-10-0-141-12 kubenswrapper[2586]: W1029 23:13:04.770182    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6957c0_aaa9_449b_b5be_4f29212f40ce.slice/crio-39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280 WatchSource:0}: Error finding container 39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280: Status 404 returned error can't find the container with id 39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906388    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906439    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906462    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906481    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906531    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906550    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:04.906573    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.906699    2586 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.906739    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.906726672 +0000 UTC m=+45.708721669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907037    2586 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907074    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907061494 +0000 UTC m=+45.709056487 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907111    2586 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907134    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907127492 +0000 UTC m=+45.709122486 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907182    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907174812 +0000 UTC m=+45.709169805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:04.907392 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907221    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907228    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907253    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907247597 +0000 UTC m=+45.709242590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907286    2586 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907304    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls podName:37f1ad0a-20e9-45bb-9340-37f5dc7410d3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907298647 +0000 UTC m=+45.709293640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-q47dp" (UID: "37f1ad0a-20e9-45bb-9340-37f5dc7410d3") : secret "samples-operator-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907342    2586 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907348    2586 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:13:04.907896 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:04.907365    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:12.907359845 +0000 UTC m=+45.709354838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:13:05.001482 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.001357    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb" event={"ID":"e82f1b60-106c-4927-ad20-18a9cdf92709","Type":"ContainerStarted","Data":"3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad"}
Oct 29 23:13:05.003617 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.003475    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fth5d-rckp7" event={"ID":"de6957c0-aaa9-449b-b5be-4f29212f40ce","Type":"ContainerStarted","Data":"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"}
Oct 29 23:13:05.003617 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.003523    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fth5d-rckp7" event={"ID":"de6957c0-aaa9-449b-b5be-4f29212f40ce","Type":"ContainerStarted","Data":"39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280"}
Oct 29 23:13:05.006774 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.006747    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7" event={"ID":"a1839b99-bbb7-4a20-9507-7d72dbbe354d","Type":"ContainerStarted","Data":"5c380a8778225e415a8ef34c03443e426e38b86a33bf83d1c26f7ce5cfa48d34"}
Oct 29 23:13:05.006884 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.006791    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7" event={"ID":"a1839b99-bbb7-4a20-9507-7d72dbbe354d","Type":"ContainerStarted","Data":"1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0"}
Oct 29 23:13:05.007647 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.007624    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:05.007796 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.007777    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 29 23:13:05.007887 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.007832    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls podName:d42a6856-295d-4cf1-8a6e-691fcc073a34 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.00781683 +0000 UTC m=+45.809811842 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls") pod "dns-default-f8xtp" (UID: "d42a6856-295d-4cf1-8a6e-691fcc073a34") : secret "dns-default-metrics-tls" not found
Oct 29 23:13:05.009499 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.009476    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t" event={"ID":"e0347d66-da9b-4703-85f2-cfe2f25eb1d9","Type":"ContainerStarted","Data":"46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35"}
Oct 29 23:13:05.057223 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.057105    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-fth5d-rckp7" podStartSLOduration=1.05709359 podStartE2EDuration="1.05709359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:05.055103136 +0000 UTC m=+37.857098151" watchObservedRunningTime="2026-10-29 23:13:05.05709359 +0000 UTC m=+37.859088605"
Oct 29 23:13:05.100410 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.100369    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7" podStartSLOduration=2.100357721 podStartE2EDuration="2.100357721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:05.098681216 +0000 UTC m=+37.900676233" watchObservedRunningTime="2026-10-29 23:13:05.100357721 +0000 UTC m=+37.902352764"
Oct 29 23:13:05.258698 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.258538    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:05.715012 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.714914    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:05.715376 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:05.715347    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:05.715376 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.715107    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:05.715547 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.715463    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:07.715425514 +0000 UTC m=+40.517420521 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:05.715547 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.715470    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:05.715547 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:05.715529    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:07.715513623 +0000 UTC m=+40.517508633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:06.014568 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.013335    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:06.016253 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.016223    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-q9xkk" event={"ID":"4fbaebde-1120-4288-a1bc-24d3f97bf85b","Type":"ContainerStarted","Data":"3673ebdfdcb0343a2fa1260d5d23b9a8632503b6d3e48c3c06f09a5e203242a3"}
Oct 29 23:13:06.058370 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.058325    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-q9xkk" podStartSLOduration=35.560621853 podStartE2EDuration="39.05831282s" totalImagesPullingTime="3.497690967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:06.057856451 +0000 UTC m=+38.859851467" watchObservedRunningTime="2026-10-29 23:13:06.05831282 +0000 UTC m=+38.860307829"
Oct 29 23:13:06.171859 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.171829    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:06.409237 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.409210    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:06.572078 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:06.571990    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:07.021259 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.021221    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb" event={"ID":"e82f1b60-106c-4927-ad20-18a9cdf92709","Type":"ContainerStarted","Data":"127285e65d384f1c89ab8926ad400083b8323e9c042a969f19229996c0ff7b8e"}
Oct 29 23:13:07.022958 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.022921    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t" event={"ID":"e0347d66-da9b-4703-85f2-cfe2f25eb1d9","Type":"ContainerStarted","Data":"4536b4df6cb57c7fd1e10278f4cd5d459897249ec47ec8add948169761efb4a8"}
Oct 29 23:13:07.023100 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.022967    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:07.064417 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.064366    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb" podStartSLOduration=2.211305768 podStartE2EDuration="4.064348611s" totalImagesPullingTime="1.853042843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:07.062383595 +0000 UTC m=+39.864378609" watchObservedRunningTime="2026-10-29 23:13:07.064348611 +0000 UTC m=+39.866343627"
Oct 29 23:13:07.104546 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.104498    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t" podStartSLOduration=2.240772426 podStartE2EDuration="4.104481685s" totalImagesPullingTime="1.863709259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:07.102849914 +0000 UTC m=+39.904844929" watchObservedRunningTime="2026-10-29 23:13:07.104481685 +0000 UTC m=+39.906476701"
Oct 29 23:13:07.104546 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.104537    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4blvf_e5f497d2-8302-4707-b479-4dca872829e3/dns-node-resolver/0.log"
Oct 29 23:13:07.431466 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.431436    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:07.431635 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.431496    2586 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:07.431882 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.431865    2586 scope.go:122] "RemoveContainer" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2"
Oct 29 23:13:07.432143 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:07.432119    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-7v65t_openshift-console-operator(c765f10e-91f1-4c9f-9ffd-2740b2bde8b9)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" podUID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9"
Oct 29 23:13:07.647513 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.647481    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:07.735410 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.735330    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:07.735565 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:07.735444    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:07.735565 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:07.735520    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.735503042 +0000 UTC m=+44.537498054 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:07.735565 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.735545    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:07.735754 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:07.735673    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:07.735754 ip-10-0-141-12 kubenswrapper[2586]: E1029 23:13:07.735716    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.735702442 +0000 UTC m=+44.537697436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:07.790229 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.790195    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:07.929763 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:07.929732    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:08.069169 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:08.069088    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:08.105042 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:08.105018    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9m5t_d8081555-b5de-4ffa-bbe7-e2931b269310/node-ca/0.log"
Oct 29 23:13:08.251159 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:08.251117    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:08.389840 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:08.389815    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:08.534488 ip-10-0-141-12 kubenswrapper[2586]: I1029 23:13:08.534471    2586 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:08.854350 ip-10-0-141-12 systemd[1]: Stopping Kubernetes Kubelet...
Oct 29 23:13:08.868331 ip-10-0-141-12 systemd[1]: kubelet.service: Deactivated successfully.
Oct 29 23:13:08.868497 ip-10-0-141-12 systemd[1]: Stopped Kubernetes Kubelet.
Oct 29 23:13:08.868548 ip-10-0-141-12 systemd[1]: kubelet.service: Consumed 3.086s CPU time.
Oct 29 23:13:08.883521 ip-10-0-141-12 systemd[1]: Starting Kubernetes Kubelet...
Oct 29 23:13:08.971939 ip-10-0-141-12 kubenswrapper[5111]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:08.971939 ip-10-0-141-12 kubenswrapper[5111]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 29 23:13:08.971939 ip-10-0-141-12 kubenswrapper[5111]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:08.971939 ip-10-0-141-12 kubenswrapper[5111]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 29 23:13:08.974756 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974683    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.974756 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974754    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.974756 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974758    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.974756 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974761    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974765    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974769    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974772    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974775    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974778    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974780    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974783    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974786    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974789    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974791    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974794    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974797    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974800    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974803    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974805    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974808    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974811    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974813    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974816    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.974925 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974819    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974821    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974824    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974827    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974830    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974832    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974835    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974840    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974843    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974846    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974849    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974851    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974854    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974857    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974860    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974862    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974865    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974868    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974871    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.975414 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974874    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974876    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974879    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974882    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974885    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974887    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974890    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974892    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974895    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974898    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974900    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974903    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974906    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974908    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974911    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974914    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974916    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974919    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974923    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.975870 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974927    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974930    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974933    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974936    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974939    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974942    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974945    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974948    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974952    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974954    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974957    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974960    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974962    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974965    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974969    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974988    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974993    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974997    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.974999    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.976347 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975002    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975005    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975008    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975012    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975014    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975017    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975020    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975023    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975027    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975031    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975035    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975040    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975044    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975048    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975051    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975053    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975056    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975059    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975063    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975065    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.976814 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975068    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975071    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975073    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975077    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975080    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975082    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975818    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975824    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975827    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975830    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975833    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975836    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975839    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975841    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975844    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975846    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975851    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975854    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975856    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975859    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.977322 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975861    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975865    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975867    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975870    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975873    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975876    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975879    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975882    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975885    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975887    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975890    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975894    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975896    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975899    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975901    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975904    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975907    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975910    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975912    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.977803 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975915    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975917    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975920    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975923    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975925    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975928    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975930    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975933    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975936    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975939    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975942    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975945    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975948    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975950    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975953    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975956    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975958    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975961    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975965    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975969    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.978305 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975991    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.975996    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976000    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976004    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976008    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976011    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976014    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976016    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976019    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976022    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976025    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976028    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976031    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976034    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976037    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976039    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976041    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976044    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976047    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976050    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.978784 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976052    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976055    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976057    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976061    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976063    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976066    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976068    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976071    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976074    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976076    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976079    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976081    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976084    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976087    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976089    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976092    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976095    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976098    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976100    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.979278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976103    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976106    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976111    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976114    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976116    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976119    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976122    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976125    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976127    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976130    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976133    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976137    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976141    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976145    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976629    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976635    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976642    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976646    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976649    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.979780 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976652    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976655    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976659    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976662    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976664    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976667    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976670    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976672    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976675    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976678    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976681    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976684    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976686    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976690    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976694    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976698    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976701    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976703    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976706    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.980250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976709    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976711    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976714    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976717    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976719    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976722    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976724    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976727    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976729    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976743    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976746    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976750    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976752    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976755    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976758    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976762    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976766    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976769    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976772    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.980710 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976775    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976778    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976781    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976784    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976786    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976789    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976792    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976794    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976798    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976801    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976804    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976807    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976810    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976812    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976815    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976818    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976821    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976824    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976826    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976829    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.981191 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976832    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976835    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976837    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976844    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976846    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976849    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976852    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976855    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976858    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976860    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976863    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976865    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976868    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976871    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976873    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976876    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976879    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976882    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976885    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.981663 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976888    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976890    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976893    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976895    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976898    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976900    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976903    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976906    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976908    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976911    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976913    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976916    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976919    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976923    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976927    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976931    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976937    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976941    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976945    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976950    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.982141 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976954    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976958    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976962    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.976966    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977089    5111 flags.go:64] FLAG: --address="0.0.0.0"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977100    5111 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977109    5111 flags.go:64] FLAG: --anonymous-auth="true"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977117    5111 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977124    5111 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977128    5111 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977132    5111 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977137    5111 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977140    5111 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977143    5111 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977146    5111 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977150    5111 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977153    5111 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977156    5111 flags.go:64] FLAG: --cgroup-root=""
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977159    5111 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977162    5111 flags.go:64] FLAG: --client-ca-file=""
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977165    5111 flags.go:64] FLAG: --cloud-provider="external"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977168    5111 flags.go:64] FLAG: --cluster-dns="[]"
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977172    5111 flags.go:64] FLAG: --cluster-domain=""
Oct 29 23:13:08.982619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977175    5111 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977178    5111 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977181    5111 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977185    5111 flags.go:64] FLAG: --container-log-max-files="5"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977189    5111 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977193    5111 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977196    5111 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977200    5111 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977203    5111 flags.go:64] FLAG: --contention-profiling="false"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977206    5111 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977209    5111 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977213    5111 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977216    5111 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977220    5111 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977223    5111 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977226    5111 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977229    5111 flags.go:64] FLAG: --enable-load-reader="false"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977233    5111 flags.go:64] FLAG: --enable-server="true"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977236    5111 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977240    5111 flags.go:64] FLAG: --event-burst="100"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977243    5111 flags.go:64] FLAG: --event-qps="50"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977246    5111 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977250    5111 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977253    5111 flags.go:64] FLAG: --eviction-hard=""
Oct 29 23:13:08.983180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977257    5111 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977260    5111 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977263    5111 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977266    5111 flags.go:64] FLAG: --eviction-soft=""
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977269    5111 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977272    5111 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977276    5111 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977279    5111 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977282    5111 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977285    5111 flags.go:64] FLAG: --fail-swap-on="true"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977288    5111 flags.go:64] FLAG: --feature-gates=""
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977292    5111 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977295    5111 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977299    5111 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977302    5111 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977305    5111 flags.go:64] FLAG: --healthz-port="10248"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977309    5111 flags.go:64] FLAG: --help="false"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977311    5111 flags.go:64] FLAG: --hostname-override="ip-10-0-141-12.ec2.internal"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977315    5111 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977318    5111 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977321    5111 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977325    5111 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977328    5111 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977332    5111 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 29 23:13:08.983757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977335    5111 flags.go:64] FLAG: --image-service-endpoint=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977342    5111 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977345    5111 flags.go:64] FLAG: --kube-api-burst="100"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977348    5111 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977351    5111 flags.go:64] FLAG: --kube-api-qps="50"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977354    5111 flags.go:64] FLAG: --kube-reserved=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977357    5111 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977360    5111 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977363    5111 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977366    5111 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977369    5111 flags.go:64] FLAG: --lock-file=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977372    5111 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977375    5111 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977378    5111 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977384    5111 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977387    5111 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977390    5111 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977393    5111 flags.go:64] FLAG: --logging-format="text"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977396    5111 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977399    5111 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977402    5111 flags.go:64] FLAG: --manifest-url=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977406    5111 flags.go:64] FLAG: --manifest-url-header=""
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977410    5111 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977413    5111 flags.go:64] FLAG: --max-open-files="1000000"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977417    5111 flags.go:64] FLAG: --max-pods="110"
Oct 29 23:13:08.984388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977420    5111 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977423    5111 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977428    5111 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977431    5111 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977434    5111 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977437    5111 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977440    5111 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977446    5111 flags.go:64] FLAG: --node-status-max-images="50"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977451    5111 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977454    5111 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977457    5111 flags.go:64] FLAG: --pod-cidr=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977460    5111 flags.go:64] FLAG: --pod-manifest-path=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977463    5111 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977466    5111 flags.go:64] FLAG: --pods-per-core="0"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977469    5111 flags.go:64] FLAG: --port="10250"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977472    5111 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977475    5111 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ab0571271bd57984"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977478    5111 flags.go:64] FLAG: --qos-reserved=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977481    5111 flags.go:64] FLAG: --read-only-port="10255"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977485    5111 flags.go:64] FLAG: --register-node="true"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977488    5111 flags.go:64] FLAG: --register-with-taints=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977492    5111 flags.go:64] FLAG: --registry-burst="10"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977495    5111 flags.go:64] FLAG: --registry-qps="5"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977497    5111 flags.go:64] FLAG: --reserved-cpus=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977500    5111 flags.go:64] FLAG: --reserved-memory=""
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977504    5111 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 29 23:13:08.984997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977507    5111 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977510    5111 flags.go:64] FLAG: --rotate-certificates="false"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977513    5111 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977516    5111 flags.go:64] FLAG: --runonce="false"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977519    5111 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977522    5111 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977525    5111 flags.go:64] FLAG: --seccomp-default="false"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977528    5111 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977531    5111 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977535    5111 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977538    5111 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977541    5111 flags.go:64] FLAG: --storage-driver-password="root"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977544    5111 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977547    5111 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977551    5111 flags.go:64] FLAG: --storage-driver-user="root"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977554    5111 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977557    5111 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977560    5111 flags.go:64] FLAG: --system-cgroups=""
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977563    5111 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977568    5111 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977571    5111 flags.go:64] FLAG: --tls-cert-file=""
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977574    5111 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977578    5111 flags.go:64] FLAG: --tls-min-version=""
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977581    5111 flags.go:64] FLAG: --tls-private-key-file=""
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977584    5111 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 29 23:13:08.985621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977587    5111 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977590    5111 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977593    5111 flags.go:64] FLAG: --v="2"
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977597    5111 flags.go:64] FLAG: --version="false"
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977601    5111 flags.go:64] FLAG: --vmodule=""
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977605    5111 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.977608    5111 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977731    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977734    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977737    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977740    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977742    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977745    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977748    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977750    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977753    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977756    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977759    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977762    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977764    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977767    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.986216 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977771    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977774    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977776    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977779    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977781    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977784    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977787    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977790    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977792    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977795    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977797    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977800    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977803    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977806    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977808    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977811    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977815    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977818    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977820    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977823    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.986718 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977825    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977828    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977832    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977835    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977837    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977840    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977843    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977845    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977848    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977851    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977854    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977857    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977861    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977863    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977866    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977869    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977871    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977874    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977877    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977879    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.987202 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977882    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977885    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977887    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977890    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977892    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977895    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977897    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977900    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977903    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977905    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977908    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977910    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977913    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977916    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977920    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977923    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977925    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977929    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977931    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.987739 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977935    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977939    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977943    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977947    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977950    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977955    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977957    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977960    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977963    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977966    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.977968    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978040    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978043    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978046    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978049    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978051    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978054    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978056    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978059    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.988490 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978062    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978064    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978067    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978070    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978072    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978075    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978077    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978080    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978084    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978088    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978092    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978095    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978098    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978100    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.978134    5111 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:08.989263 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978171    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978176    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978179    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978184    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978186    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978189    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978192    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978194    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978197    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978200    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978203    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978206    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978208    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978211    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978214    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978216    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978219    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978221    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978224    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978226    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.989825 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978229    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978232    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978234    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978237    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978239    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978242    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978246    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978249    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978252    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978254    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978257    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978259    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978262    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978264    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978267    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978272    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978275    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978278    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978280    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978283    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.990421 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978286    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978288    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978291    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978294    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978297    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978299    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978302    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978304    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978307    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978309    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978312    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978315    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978318    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978320    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978323    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978326    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978329    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978331    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978335    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.990940 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978338    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978341    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978343    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978346    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978348    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978351    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978354    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978357    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978361    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978364    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978367    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978369    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978372    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978375    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978379    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978383    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978387    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978390    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978394    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.991606 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978397    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978399    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978402    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978404    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978407    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978410    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978412    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978415    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978417    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978420    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978423    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978425    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978430    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978432    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978435    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978438    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978440    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978443    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978446    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.992320 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978448    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978451    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978455    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978457    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978462    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978466    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978469    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978472    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.978475    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.982299    5111 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.982314    5111 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.982333    5111 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.982338    5111 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982406    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982411    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982414    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982417    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982421    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982424    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.992785 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982427    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982430    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982433    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982436    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982439    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982442    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982444    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982448    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982451    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982453    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982456    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982459    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982461    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982464    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982467    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982470    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982472    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982482    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982486    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.993262 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982489    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982493    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982496    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982499    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982502    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982505    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982507    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982510    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982513    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982515    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982518    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982521    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982523    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982526    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982528    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982531    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982533    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982536    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982538    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982541    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.993712 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982544    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982546    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982549    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982551    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982554    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982556    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982559    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982561    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982564    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982566    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982569    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982572    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982575    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982577    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982580    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982583    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982586    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982588    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982593    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.994329 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982596    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982599    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982602    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982604    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982607    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982609    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982612    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982614    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982617    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982619    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982622    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982625    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982628    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982630    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982633    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982636    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982639    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982641    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982644    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982646    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.994810 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982649    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982651    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982654    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982657    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982660    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982663    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982665    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982668    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982671    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982673    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982676    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982679    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982681    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982684    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982686    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982689    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982691    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982694    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982697    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982699    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.995426 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982702    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982704    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.982730    5111 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982894    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982900    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982903    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982906    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982909    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982912    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982915    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982918    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982920    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982923    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982925    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982928    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982931    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:08.996102 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982934    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982936    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982940    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982942    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982945    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982948    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982951    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982953    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982956    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982959    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982961    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982964    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982967    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982970    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982987    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982991    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982996    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.982998    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983003    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:08.996727 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983007    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983010    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983013    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983015    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983018    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983021    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983024    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983026    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983029    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983032    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983034    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983037    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983040    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983047    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983050    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983052    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983056    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983058    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983061    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983064    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:08.997291 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983067    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983070    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983073    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983075    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983078    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983081    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983084    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983086    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983089    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983092    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983094    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983097    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983099    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983102    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983105    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983107    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983110    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983112    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983115    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983117    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:08.997776 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983120    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983123    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983125    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983128    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983130    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983135    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983137    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983140    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983143    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983146    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983149    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983151    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983154    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983157    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983159    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983163    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983167    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983171    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983174    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.998356 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983177    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983179    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983182    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983185    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983188    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983191    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983193    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983196    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983199    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983202    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983205    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983207    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983210    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983213    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983215    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.983239    5111 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 29 23:13:08.998846 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983279    5111 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983284    5111 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983289    5111 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983293    5111 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983296    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983299    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983301    5111 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983304    5111 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983308    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983310    5111 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983313    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983316    5111 feature_gate.go:386] unrecognized feature gate: Example2
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983318    5111 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983322    5111 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983326    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983329    5111 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983332    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983335    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983338    5111 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 29 23:13:08.999243 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983341    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983343    5111 feature_gate.go:386] unrecognized feature gate: Example
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983346    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983349    5111 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983351    5111 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983354    5111 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983356    5111 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983359    5111 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983362    5111 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983364    5111 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983367    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983370    5111 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983372    5111 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983375    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983378    5111 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983382    5111 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983384    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983387    5111 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983390    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 29 23:13:08.999695 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983393    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983395    5111 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983404    5111 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983407    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983410    5111 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983412    5111 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983415    5111 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983417    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983420    5111 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983422    5111 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983425    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983428    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983430    5111 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983433    5111 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983436    5111 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983439    5111 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983441    5111 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983444    5111 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983446    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983449    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 29 23:13:09.000193 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983451    5111 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983454    5111 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983456    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983459    5111 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983461    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983465    5111 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983469    5111 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983472    5111 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983476    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983478    5111 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983481    5111 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983483    5111 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983486    5111 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983489    5111 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983491    5111 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983499    5111 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983502    5111 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983505    5111 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983507    5111 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 29 23:13:09.000674 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983510    5111 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983512    5111 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983515    5111 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983517    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983520    5111 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983523    5111 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983525    5111 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983528    5111 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983531    5111 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983533    5111 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983536    5111 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983539    5111 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983542    5111 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983546    5111 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983550    5111 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983559    5111 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983563    5111 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983567    5111 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983571    5111 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983575    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 29 23:13:09.001156 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983577    5111 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983581    5111 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983584    5111 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983587    5111 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983590    5111 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983594    5111 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983598    5111 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983602    5111 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:08.983606    5111 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.983686    5111 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.984712    5111 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.984777    5111 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.984959    5111 server.go:1014] "Starting client certificate rotation"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.985141    5111 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.985249    5111 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:27 +0000 UTC" deadline="2028-08-12 21:44:26.240493234 +0000 UTC"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.985314    5111 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15670h31m17.255181347s"
Oct 29 23:13:09.001635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.985560    5111 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:13:09.002023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.986745    5111 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 29 23:13:09.002023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.989494    5111 log.go:25] "Validated CRI v1 runtime API"
Oct 29 23:13:09.002023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.991169    5111 log.go:25] "Validated CRI v1 image API"
Oct 29 23:13:09.002023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.991889    5111 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 29 23:13:09.002023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.993901    5111 fs.go:135] Filesystem UUIDs: map[24fb1fde-7b0c-4328-afa4-cd4cd3b36d3c:/dev/nvme0n1p3 3cfa7feb-b3bb-47df-8bee-348f92be40fe:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 29 23:13:09.002174 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:08.993921    5111 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0/userdata/shm major:0 minor:341 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/35489098b1a4431ee682829332159e879813364e8b0050e43b26fd8151ebc2af/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/35489098b1a4431ee682829332159e879813364e8b0050e43b26fd8151ebc2af/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280/userdata/shm major:0 minor:348 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad/userdata/shm major:0 minor:338 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35/userdata/shm major:0 minor:336 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/63d36568ecfae10c9e52634832bed8893c546ca332fcf6787353f60eab635a0b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/63d36568ecfae10c9e52634832bed8893c546ca332fcf6787353f60eab635a0b/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc/userdata/shm major:0 minor:97 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe/userdata/shm major:0 minor:229 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d7333f0931e99cf7615f2a785269d62f1853fb12dba98bddf6c82efc1cf6d6e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d7333f0931e99cf7615f2a785269d62f1853fb12dba98bddf6c82efc1cf6d6e/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/90378bcab5f1bb1f2b080b3f0d98d0f5bd275f77c3d953da15dfb8398e2dc8ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/90378bcab5f1bb1f2b080b3f0d98d0f5bd275f77c3d953da15dfb8398e2dc8ea/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/994759f998f6643cbc441a452fb4e78c956bbd1da4a05c8221f4023dfe8a8907/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/994759f998f6643cbc441a452fb4e78c956bbd1da4a05c8221f4023dfe8a8907/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dde2fd0046a160a1605b25418199082b4ddfe15cc1df3aa54cbe0e4675ff2d71/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dde2fd0046a160a1605b25418199082b4ddfe15cc1df3aa54cbe0e4675ff2d71/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0641c627-1da8-47bc-bca1-04ade039e493/volumes/kubernetes.io~projected/kube-api-access-qwp8j:{mountpoint:/var/lib/kubelet/pods/0641c627-1da8-47bc-bca1-04ade039e493/volumes/kubernetes.io~projected/kube-api-access-qwp8j major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22482cc8-beda-4302-b88e-fed36f1bb7db/volumes/kubernetes.io~projected/kube-api-access-wvxnl:{mountpoint:/var/lib/kubelet/pods/22482cc8-beda-4302-b88e-fed36f1bb7db/volumes/kubernetes.io~projected/kube-api-access-wvxnl major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e/volumes/kubernetes.io~projected/kube-api-access-7g6mc:{mountpoint:/var/lib/kubelet/pods/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e/volumes/kubernetes.io~projected/kube-api-access-7g6mc major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/37f1ad0a-20e9-45bb-9340-37f5dc7410d3/volumes/kubernetes.io~projected/kube-api-access-hjv9k:{mountpoint:/var/lib/kubelet/pods/37f1ad0a-20e9-45bb-9340-37f5dc7410d3/volumes/kubernetes.io~projected/kube-api-access-hjv9k major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fbaebde-1120-4288-a1bc-24d3f97bf85b/volumes/kubernetes.io~projected/kube-api-access-zm68z:{mountpoint:/var/lib/kubelet/pods/4fbaebde-1120-4288-a1bc-24d3f97bf85b/volumes/kubernetes.io~projected/kube-api-access-zm68z major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/kube-api-access-clfb4:{mountpoint:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/kube-api-access-clfb4 major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7/volumes/kubernetes.io~projected/kube-api-access-6mfnd:{mountpoint:/var/lib/kubelet/pods/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7/volumes/kubernetes.io~projected/kube-api-access-6mfnd major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/61f54971-c57c-405c-93a8-19fcc3e50908/volumes/kubernetes.io~projected/kube-api-access-ss25j:{mountpoint:/var/lib/kubelet/pods/61f54971-c57c-405c-93a8-19fcc3e50908/volumes/kubernetes.io~projected/kube-api-access-ss25j major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6466c095-0df6-4688-abc3-2f6f212471a0/volumes/kubernetes.io~projected/kube-api-access-rwlfl:{mountpoint:/var/lib/kubelet/pods/6466c095-0df6-4688-abc3-2f6f212471a0/volumes/kubernetes.io~projected/kube-api-access-rwlfl major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7bd53292-a0c6-41a8-8a39-41a27853cb73/volumes/kubernetes.io~projected/kube-api-access-whdld:{mountpoint:/var/lib/kubelet/pods/7bd53292-a0c6-41a8-8a39-41a27853cb73/volumes/kubernetes.io~projected/kube-api-access-whdld major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7e1cae4c-99a8-4eb9-a957-1b38f9460e55/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/7e1cae4c-99a8-4eb9-a957-1b38f9460e55/volumes/kubernetes.io~secret/agent-certs major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~projected/kube-api-access-g259q:{mountpoint:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~projected/kube-api-access-g259q major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/86fa6768-086e-4434-9d30-cc860a28acc6/volumes/kubernetes.io~projected/kube-api-access-7mn8r:{mountpoint:/var/lib/kubelet/pods/86fa6768-086e-4434-9d30-cc860a28acc6/volumes/kubernetes.io~projected/kube-api-access-7mn8r major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3/volumes/kubernetes.io~projected/kube-api-access-mmdn8:{mountpoint:/var/lib/kubelet/pods/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3/volumes/kubernetes.io~projected/kube-api-access-mmdn8 major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~projected/kube-api-access-75xdt:{mountpoint:/var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~projected/kube-api-access-75xdt major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~secret/signing-key major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aae915ed-0747-410a-acec-a18068a98efe/volumes/kubernetes.io~projected/kube-api-access-rxwxd:{mountpoint:/var/lib/kubelet/pods/aae915ed-0747-410a-acec-a18068a98efe/volumes/kubernetes.io~projected/kube-api-access-rxwxd major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/adbe2362-88df-49bd-9692-e1311e28c396/volumes/kubernetes.io~projected/kube-api-access-9grgn:{mountpoint:/var/lib/kubelet/pods/adbe2362-88df-49bd-9692-e1311e28c396/volumes/kubernetes.io~projected/kube-api-access-9grgn major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~projected/kube-api-access-cl9zc:{mountpoint:/var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~projected/kube-api-access-cl9zc major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d42a6856-295d-4cf1-8a6e-691fcc073a34/volumes/kubernetes.io~projected/kube-api-access-zqllh:{mountpoint:/var/lib/kubelet/pods/d42a6856-295d-4cf1-8a6e-691fcc073a34/volumes/kubernetes.io~projected/kube-api-access-zqllh major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~projected/kube-api-access-8dnjr:{mountpoint:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~projected/kube-api-access-8dnjr major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/default-certificate major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/stats-auth major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d8081555-b5de-4ffa-bbe7-e2931b269310/volumes/kubernetes.io~projected/kube-api-access-6bkm9:{mountpoint:/var/lib/kubelet/pods/d8081555-b5de-4ffa-bbe7-e2931b269310/volumes/kubernetes.io~projected/kube-api-access-6bkm9 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~projected/kube-api-access-cvs8h:{mountpoint:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~projected/kube-api-access-cvs8h major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/de6957c0-aaa9-449b-b5be-4f29212f40ce/volumes/kubernetes.io~projected/kube-api-access-c254g:{mountpoint:/var/lib/kubelet/pods/de6957c0-aaa9-449b-b5be-4f29212f40ce/volumes/kubernetes.io~projected/kube-api-access-c254g major:0 minor:343 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e0347d66-da9b-4703-85f2-cfe2f25eb1d9/volumes/kubernetes.io~projected/kube-api-access-5x2bw:{mountpoint:/var/lib/kubelet/pods/e0347d66-da9b-4703-85f2-cfe2f25eb1d9/volumes/kubernetes.io~projected/kube-api-access-5x2bw major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e29dccc7-5622-4583-b74e-220df6ff5a34/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/e29dccc7-5622-4583-b74e-220df6ff5a34/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~projected/kube-api-access-7rpww:{mountpoint:/var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~projected/kube-api-access-7rpww major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~secret/serving-cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e5f497d2-8302-4707-b479-4dca872829e3/volumes/kubernetes.io~projected/kube-api-access-nz42q:{mountpoint:/var/lib/kubelet/pods/e5f497d2-8302-4707-b479-4dca872829e3/volumes/kubernetes.io~projected/kube-api-access-nz42q major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~projected/kube-api-access-f2b6r:{mountpoint:/var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~projected/kube-api-access-f2b6r major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e82f1b60-106c-4927-ad20-18a9cdf92709/volumes/kubernetes.io~projected/kube-api-access-gsbhp:{mountpoint:/var/lib/kubelet/pods/e82f1b60-106c-4927-ad20-18a9cdf92709/volumes/kubernetes.io~projected/kube-api-access-gsbhp major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f5c0147f-93d0-431f-a294-93c26838eff3/volumes/kubernetes.io~projected/kube-api-access-8k4nt:{mountpoint:/var/lib/kubelet/pods/f5c0147f-93d0-431f-a294-93c26838eff3/volumes/kubernetes.io~projected/kube-api-access-8k4nt major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~projected/kube-api-access-89dfq:{mountpoint:/var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~projected/kube-api-access-89dfq major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~secret/cert major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/kube-api-access-5llz7:{mountpoint:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/kube-api-access-5llz7 major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~projected/kube-api-access-2db6t:{mountpoint:/var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~projected/kube-api-access-2db6t major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~secret/serving-cert major:0 minor:205 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/b18d6369c009e2ff429f53f901c3a1ad12f7a525d3629dbd68abdcc6b7ed2f65/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/07bf7d29c48ef767ef41190c88edff57b2d0808797f74b3e01273c016424e0db/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/fb79124ac4b1319333e9e0d8e0f60e283281718bdd06db05d134b5bcf219c8c3/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/ecb3c3871fe775f4f1fbf975fd7bb86a36adebacc930b9b473f3ccba7bf4983c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/5bf174b585ff40e1a3688665b93241410d6d4a1ba73c923e7aa12cc8c54ccef9/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/9247ab1016ba507140262bf451a47467901bb90c3528734aad9bd31b5b9bd689/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/7a36461bae4d5017321abf63ef54c12c28c9f1d94a6a57cf945c95baa804e096/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/7417d065b4b4b199c5266b2325e7ba719d02097f98aa2113e2e4645d9f0c135b/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/a6b94cde66647a8d23c7c304c5542a99c426a3e333f9d1838a1748d1691fd6cb/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/c0e97456a8bb9d457588d15506f5b7ed08a91608defcc9d87729dca7bcbb6b35/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-140:{mountpoint:/var/lib/containers/storage/overlay/6d79dbf905de75eccac74b176d8288b5c82976e50a20d0055eebb66c2d07ed1b/merged major:0 minor:140 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/13c2c52f3a11a25c38dbb169ed0bb4a91234477b44793c9a7e557066e7dde830/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/5f71df6393b32d4951fb93eb9af6cf7caf6587b21d7a9752238cdcb700681670/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/a62c9e36668e09fd966e6389ec7a904ee00a08d33c2309695565310ce76c1a7b/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/d1004d738b4b104fb623e41bb042fcebfe6b6b3fec5562fef7b24097b0c3d979/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/00dedec0c96fde4f3680eed191ec3a59b870d69fe4aae7016290d667998703a3/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/334e531a44a2384deb54f7121175860ca824bd7196af50fb92d847796cae204f/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/7c85d826af4f6415310dacae693110672f0cb318805fc6557f63af1ddd8bcf96/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/19285bfc1aa1047626ead8d141476a0bebaad9bd1a6968a3a1e00c61d26b16be/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/a24d0616d9666d66b7876f31cea1cb0143dd231ba402273b88fc7117e3c09e22/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/c5aae9948737b64891c08d6021e861d95abb1b11ca867b884706871b2e2847ee/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/57d10568bc2c6c1f452d38a303bf7f0f21be4d1d7b3ad8f733b917af2a013200/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/5239f2dc37391331c96c681988080098b3a47d715c11b006ab1b101d08fb51f4/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/2fcc22803ef55e1342afdaf0b7108222b209f75f7271fabc1c010c6a0ec8a52f/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/196db464893a45212f4d24006e4302e68155bfbef14c0e297981b622353fab20/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/2c22ab5e410df986f042e413cb84b2cdd995da2bb5dd994bf47403ff8c2c35f6/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/dabf2000c0a4ad9b58e3fbdacddedf519d30638a26f39426dddd971f580b92d3/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/e632d08cff1ee733a2c8e07e38d9e90fece59f33f6a087c89e929fb3139a1fd4/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/326667882f9d374e469476c8a24d4421e47caede6e68e3bc184bef7f962fb527/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/b4acce89304bf4ce81a8087678a55a0141f5e11a301294d4a10d10d90816de01/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/829bc5b194ea949dbc9ce0c84434ae067bfc8c78a41b73522a4fc737d52aff5e/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-267:{mountpoint:/var/lib/containers/storage/overlay/bc798aa0ccacbe2a299de65ae4ca38d44021ada719d3dedafbf56002cc4d69b5/merged major:0 minor:267 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/fe29d900836d237db0ff0c827d9d1e6aa03bb44f4d76ae2bd2ddcc3425663389/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/12dbb23ff578c350bca8b2c2358a2cad4d5e8f4e3e80818d8d1fcf9b73aad2dc/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/7e656a71101e316dbaaeb3e93e4c795ee3f0fe2e07dad9919002b6caa0170911/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/9b8d6e1fbd9dc7842d42810585a7dd84633c04decb47b8b28fbc4836ddd594bd/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/0be020c0300fda36b78659eaf98d84e497e092a5f65206c8ec5135df1e1a33a7/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/83be6c039b49374343d4e8561c410d92f3fb7b5dbde495e03740224fd7b6a867/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/9813db848221bb9f6afb65a7600ea9fb6132ac46972f6979feabfa43d7258581/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/320ad434b8c344562c80223f24fe126176c72fce27381467bdc6cb9a872586c9/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-308:{mountpoint:/var/lib/containers/storage/overlay/62793f8487ae7f4345b75551e198f48102f6781e5c66dd67a0fb38e78838016f/merged major:0 minor:308 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/2df82dbd491aa4bb015678f041fc924e7b58125ed3391313cee96e8680788f14/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/3e9a744380099f826db333134a0724bc40c920991a2509c4e5448f61c5e1096a/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/782403a868ac73dad7237ac90d93462211857587103678d3578c36b3d0fea065/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/29eb5b879704308def71418bd83db1ee8062ac8e2df3274831063f517cc19609/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/3e523017c4f37a08ef476738d7ea4545d5e445f2911325b9197c008b9652a835/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/959addd1cfeca847b39e550a4f6df5094c5157b92d98a23eac8119ba0737f97c/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-360:{mountpoint:/var/lib/containers/storage/overlay/d7deadf0bc56ad7bb484dd4815c2df9bbe341c2a16a075c1e98ff480f9c0c6a6/merged major:0 minor:360 fsType:overlay blockSize:0} overlay_0-362:{mountpoint:/var/lib/containers/storage/overlay/61a93fbaa249db26baaead7108bddda1ea2462f226978ab3b423395e7a457038/merged major:0 minor:362 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/067105c2d6642e41f80e29ba94c1c3e480244ea4990ff4ea8c92bd7638a8e4a1/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/82fc06cb322dd162c83c23591f7609eb149cacff66fef0e97defcf69916397ea/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/d18671023145679415292cedcab6a9ccde8c3ac92011eaaee88c500b79492e71/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/e46ba8b0d93b1c84dd801cacaa2e349f561a88ceca813dc61d145c2fc0137771/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/d0a921518f77fcd2666cb879f6a4f7a02933a629658021703b62cba4cc4776c2/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/f5998afb6471ca439ec3e3ee46ba8632fd8ee2a22d815111ffe08dae608f46b2/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/5d9265abebd71311abc95ceb073f15c67e15906c68c29fc95946b1c56727974e/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/0c4431ce1229e7490a4188745e9550594d98171163e6b045c43fe8843710c9d4/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-81:{mountpoint:/var/lib/containers/storage/overlay/c75b841f7f61371541fb64df5c2a7df5243e022c80180dc0eb8cb07c995d45d4/merged major:0 minor:81 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/f7c932e647d0bee92b43cd0cd0c48c9e34f296703398160343d278d6cf4f996c/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/99a6cd199360d75ff34c5949a3209f2cf4f4d36c35bad08f83d963c6591a8d17/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/058734013a45886f2d97d808f818ef5fda4663fa41e4edde24f2f8520e066a98/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/b16f1f2b9184949e1467358d72802c55cd0d77f6283c15344ca83781e1dc0a41/merged major:0 minor:94 fsType:overlay blockSize:0}]
Oct 29 23:13:09.002643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.001874    5111 manager.go:217] Machine: {Timestamp:2026-10-29 23:13:09.001427343 +0000 UTC m=+0.087710475 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099253 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec226a6734fb29efdb5b6c4ccfdc1a69 SystemUUID:ec226a67-34fb-29ef-db5b-6c4ccfdc1a69 BootID:1b67e816-e9b5-49fe-ad83-890f187f3cd2 Filesystems:[{Device:/var/lib/kubelet/pods/61f54971-c57c-405c-93a8-19fcc3e50908/volumes/kubernetes.io~projected/kube-api-access-ss25j DeviceMajor:0 DeviceMinor:67 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e29dccc7-5622-4583-b74e-220df6ff5a34/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:295 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3/volumes/kubernetes.io~projected/kube-api-access-mmdn8 DeviceMajor:0 DeviceMinor:62 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9017733403dbe01779c7d8f3fead5e5044a152244651eac272fc61f46ad4516a/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:204 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:334 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d7333f0931e99cf7615f2a785269d62f1853fb12dba98bddf6c82efc1cf6d6e/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:231 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/abb4e8d62a20fd9e5ce1d7f21ec6c9566a4731bc2a8ccc16ed4c22ebf8357eb0/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/763ac527689bc61716f32a24fc26fbd8af88ad4d9d9af4f418a088115a2d6dcc/userdata/shm DeviceMajor:0 DeviceMinor:97 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:202 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d60dd561ec02e592e2c2b37bc9a3a2c4e5ba74cfc77104d0484cd7446e7fe96a/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1fc1d71368c5ab008907d171bd117ff3bf9330d346576b901da3759dc9d09ea0/userdata/shm DeviceMajor:0 DeviceMinor:341 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-362 DeviceMajor:0 DeviceMinor:362 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:197 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/22482cc8-beda-4302-b88e-fed36f1bb7db/volumes/kubernetes.io~projected/kube-api-access-wvxnl DeviceMajor:0 DeviceMinor:340 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:205 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/7e1cae4c-99a8-4eb9-a957-1b38f9460e55/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3d3f3cb3ba583f5359cbeb07d3139030cc81437ea88f39d8ba2805d71480dbad/userdata/shm DeviceMajor:0 DeviceMinor:338 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f63a0452-9661-4fbb-9396-9ac7b5524dd3/volumes/kubernetes.io~projected/kube-api-access-89dfq DeviceMajor:0 DeviceMinor:236 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~projected/kube-api-access-cvs8h DeviceMajor:0 DeviceMinor:65 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/994759f998f6643cbc441a452fb4e78c956bbd1da4a05c8221f4023dfe8a8907/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d8081555-b5de-4ffa-bbe7-e2931b269310/volumes/kubernetes.io~projected/kube-api-access-6bkm9 DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~projected/kube-api-access-g259q DeviceMajor:0 DeviceMinor:66 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/kube-api-access-clfb4 DeviceMajor:0 DeviceMinor:223 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6dcf9ec097b63f7273b46967046cef54408e4b2bce5989616790be0201bb4740/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e3586f85e12a17773e96e6e5d2429bfdec51ccf120b30a59ab1e8d80d2fa099/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/37c3d4dfe1cde3c210c22aaf606eb2568bbb139b09518bc782259e44be57bb06/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d7081746-11b4-4ea2-b641-2fc60782a2cf/volumes/kubernetes.io~projected/kube-api-access-8dnjr DeviceMajor:0 DeviceMinor:211 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~projected/kube-api-access-cl9zc DeviceMajor:0 DeviceMinor:218 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e82f1b60-106c-4927-ad20-18a9cdf92709/volumes/kubernetes.io~projected/kube-api-access-gsbhp DeviceMajor:0 DeviceMinor:328 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d2a14a17c901901e16f012e7b490f6f19257aa4d7d6b5a52d71c69ea7df3ffe/userdata/shm DeviceMajor:0 DeviceMinor:229 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e0347d66-da9b-4703-85f2-cfe2f25eb1d9/volumes/kubernetes.io~projected/kube-api-access-5x2bw DeviceMajor:0 DeviceMinor:333 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/de6957c0-aaa9-449b-b5be-4f29212f40ce/volumes/kubernetes.io~projected/kube-api-access-c254g DeviceMajor:0 DeviceMinor:343 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f5c0147f-93d0-431f-a294-93c26838eff3/volumes/kubernetes.io~projected/kube-api-access-8k4nt DeviceMajor:0 DeviceMinor:219 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-308 DeviceMajor:0 DeviceMinor:308 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7/volumes/kubernetes.io~projected/kube-api-access-6mfnd DeviceMajor:0 DeviceMinor:213 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/7bd53292-a0c6-41a8-8a39-41a27853cb73/volumes/kubernetes.io~projected/kube-api-access-whdld DeviceMajor:0 DeviceMinor:216 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/0641c627-1da8-47bc-bca1-04ade039e493/volumes/kubernetes.io~projected/kube-api-access-qwp8j DeviceMajor:0 DeviceMinor:222 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e/volumes/kubernetes.io~projected/kube-api-access-7g6mc DeviceMajor:0 DeviceMinor:64 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-81 DeviceMajor:0 DeviceMinor:81 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/kube-api-access-5llz7 DeviceMajor:0 DeviceMinor:224 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-140 DeviceMajor:0 DeviceMinor:140 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d42a6856-295d-4cf1-8a6e-691fcc073a34/volumes/kubernetes.io~projected/kube-api-access-zqllh DeviceMajor:0 DeviceMinor:237 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ad7dc0c6e8056ccfbe23772a1547199d7dfe1f376da10b307136629e77ba466/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/35489098b1a4431ee682829332159e879813364e8b0050e43b26fd8151ebc2af/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-267 DeviceMajor:0 DeviceMinor:267 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:214 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2b7f1411aec1ece10047eaecedc5f24d1bc52efa9084a6987bba299b33009379/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/a1839b99-bbb7-4a20-9507-7d72dbbe354d/volumes/kubernetes.io~projected/kube-api-access-75xdt DeviceMajor:0 DeviceMinor:335 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-360 DeviceMajor:0 DeviceMinor:360 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/adbe2362-88df-49bd-9692-e1311e28c396/volumes/kubernetes.io~projected/kube-api-access-9grgn DeviceMajor:0 DeviceMinor:61 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/90378bcab5f1bb1f2b080b3f0d98d0f5bd275f77c3d953da15dfb8398e2dc8ea/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/46f4a9c6bccff55ad6117ac43a72efcd5abf2c7bf05f08471c57b2c9ca737a35/userdata/shm DeviceMajor:0 DeviceMinor:336 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a189a8994a9943220e9ba4843934b8e13c4a00b3dffa3aa9af2c381805da3af3/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4baecbf0649b9ccc0fa7de6881da8bf764394b364e49b6803ba9fa0723f37f69/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:203 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef1db75aad074768db43318eae76465a38c5d9f0b1258b0ab0701bbf5f46748d/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dde2fd0046a160a1605b25418199082b4ddfe15cc1df3aa54cbe0e4675ff2d71/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/37f1ad0a-20e9-45bb-9340-37f5dc7410d3/volumes/kubernetes.io~projected/kube-api-access-hjv9k DeviceMajor:0 DeviceMinor:217 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/055793bccd7c4c6d3679070d0949cb4cfca2aad8ad9e88c60f724077cfb7539d/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/86fa6768-086e-4434-9d30-cc860a28acc6/volumes/kubernetes.io~projected/kube-api-access-7mn8r DeviceMajor:0 DeviceMinor:210 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/63d36568ecfae10c9e52634832bed8893c546ca332fcf6787353f60eab635a0b/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:212 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4fbaebde-1120-4288-a1bc-24d3f97bf85b/volumes/kubernetes.io~projected/kube-api-access-zm68z DeviceMajor:0 DeviceMinor:294 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280/userdata/shm DeviceMajor:0 DeviceMinor:348 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/6466c095-0df6-4688-abc3-2f6f212471a0/volumes/kubernetes.io~projected/kube-api-access-rwlfl DeviceMajor:0 DeviceMinor:63 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e61d9ca2-dabd-4378-bee7-ecacbfd366a6/volumes/kubernetes.io~projected/kube-api-access-f2b6r DeviceMajor:0 DeviceMinor:220 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e5f497d2-8302-4707-b479-4dca872829e3/volumes/kubernetes.io~projected/kube-api-access-nz42q DeviceMajor:0 DeviceMinor:60 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aae915ed-0747-410a-acec-a18068a98efe/volumes/kubernetes.io~projected/kube-api-access-rxwxd DeviceMajor:0 DeviceMinor:215 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/fe93ab9c-ae4e-48af-a423-74e32a601101/volumes/kubernetes.io~projected/kube-api-access-2db6t DeviceMajor:0 DeviceMinor:221 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/de4fcb6f-b5d4-4806-882a-8145a71c40d3/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/e525b897-14cf-4e83-a016-60ab539f490d/volumes/kubernetes.io~projected/kube-api-access-7rpww DeviceMajor:0 DeviceMinor:207 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:055793bccd7c4c6 MacAddress:16:0b:3b:6b:68:b8 Speed:10000 Mtu:8901} {Name:1fc1d71368c5ab0 MacAddress:26:ee:75:84:64:69 Speed:10000 Mtu:8901} {Name:2b7f1411aec1ece MacAddress:e6:1a:1e:ef:10:3b Speed:10000 Mtu:8901} {Name:2e3586f85e12a17 MacAddress:f6:9d:2b:f3:22:d1 Speed:10000 Mtu:8901} {Name:35489098b1a4431 MacAddress:f6:02:9d:f8:bb:f8 Speed:10000 Mtu:8901} {Name:37c3d4dfe1cde3c MacAddress:de:25:9d:bf:c7:cc Speed:10000 Mtu:8901} {Name:39b2f08448bf356 MacAddress:5e:ba:d4:55:be:0c Speed:10000 Mtu:8901} {Name:3d3f3cb3ba583f5 MacAddress:a6:ad:41:ae:83:a0 Speed:10000 Mtu:8901} {Name:46f4a9c6bccff55 MacAddress:e2:8f:73:17:ae:0c Speed:10000 Mtu:8901} {Name:5ad7dc0c6e8056c MacAddress:32:f8:fb:f8:c0:cc Speed:10000 Mtu:8901} {Name:6dcf9ec097b63f7 MacAddress:62:4e:b0:26:54:99 Speed:10000 Mtu:8901} {Name:7d2a14a17c90190 MacAddress:66:c6:77:a0:ae:9f Speed:10000 Mtu:8901} {Name:90378bcab5f1bb1 MacAddress:e2:d8:66:ab:fd:36 Speed:10000 Mtu:8901} {Name:994759f998f6643 MacAddress:ca:c3:ca:ab:4c:fb Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f8:d0:72:1f Speed:0 Mtu:9001} {Name:br-int MacAddress:92:69:64:81:fa:b8 Speed:0 Mtu:8901} {Name:d60dd561ec02e59 MacAddress:4a:d6:69:4e:4c:79 Speed:10000 Mtu:8901} {Name:ef1db75aad07476 MacAddress:26:ad:ca:bf:14:bd Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f8:d0:72:1f Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:a6:2f:95:de:5d:9b Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:9e:cb:8e:bb:e8:f0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 29 23:13:09.002643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002233    5111 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 29 23:13:09.002643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002276    5111 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 29 23:13:09.002643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002584    5111 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002608    5111 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002733    5111 topology_manager.go:143] "Creating topology manager with none policy"
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002740    5111 container_manager_linux.go:311] "Creating device plugin manager"
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002754    5111 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002771    5111 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 29 23:13:09.002915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002790    5111 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.002928    5111 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003037    5111 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003068    5111 kubelet.go:493] "Attempting to sync node with API server"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003080    5111 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003091    5111 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003099    5111 kubelet.go:405] "Adding apiserver pod source"
Oct 29 23:13:09.003107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003108    5111 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 29 23:13:09.003764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003752    5111 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 29 23:13:09.003764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.003765    5111 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:13:09.004588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.004570    5111 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 29 23:13:09.007744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.007725    5111 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 29 23:13:09.007744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.007741    5111 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 29 23:13:09.007885 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.007851    5111 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 29 23:13:09.008072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008060    5111 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 29 23:13:09.008107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008090    5111 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 29 23:13:09.008209 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008200    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008214    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008221    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008226    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008232    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008237    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008243    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008249    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 29 23:13:09.008255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008256    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 29 23:13:09.008572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008262    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 29 23:13:09.008572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008270    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 29 23:13:09.008572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008280    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 29 23:13:09.008572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008311    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 29 23:13:09.008572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.008317    5111 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 29 23:13:09.011620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.011607    5111 server.go:1263] "Started kubelet"
Oct 29 23:13:09.011752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.011701    5111 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 29 23:13:09.011819 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.011805    5111 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 29 23:13:09.011933 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.011913    5111 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 29 23:13:09.012147 ip-10-0-141-12 systemd[1]: Started Kubernetes Kubelet.
Oct 29 23:13:09.012652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.012289    5111 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 29 23:13:09.013119 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.013099    5111 server.go:317] "Adding debug handlers to kubelet server"
Oct 29 23:13:09.014763 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014743    5111 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 29 23:13:09.014844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014774    5111 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 29 23:13:09.014844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014810    5111 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:29 +0000 UTC" deadline="2028-07-31 06:28:09.247897794 +0000 UTC"
Oct 29 23:13:09.014844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014838    5111 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15367h15m0.233063007s"
Oct 29 23:13:09.015006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014863    5111 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 29 23:13:09.015006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.014868    5111 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 29 23:13:09.015006 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.014940    5111 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-12.ec2.internal\" not found"
Oct 29 23:13:09.015145 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.015066    5111 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 29 23:13:09.018341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018303    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="adbe2362-88df-49bd-9692-e1311e28c396" volumeName="kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018344    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7081746-11b4-4ea2-b641-2fc60782a2cf" volumeName="kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018361    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e525b897-14cf-4e83-a016-60ab539f490d" volumeName="kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018376    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5f497d2-8302-4707-b479-4dca872829e3" volumeName="kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018396    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f63a0452-9661-4fbb-9396-9ac7b5524dd3" volumeName="kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018409    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018423    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22482cc8-beda-4302-b88e-fed36f1bb7db" volumeName="kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume" seLinuxMountContext=""
Oct 29 23:13:09.018437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018436    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8081555-b5de-4ffa-bbe7-e2931b269310" volumeName="kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018451    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de4fcb6f-b5d4-4806-882a-8145a71c40d3" volumeName="kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018465    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe93ab9c-ae4e-48af-a423-74e32a601101" volumeName="kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018479    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018499    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61f54971-c57c-405c-93a8-19fcc3e50908" volumeName="kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018511    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7081746-11b4-4ea2-b641-2fc60782a2cf" volumeName="kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018524    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018541    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe93ab9c-ae4e-48af-a423-74e32a601101" volumeName="kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018554    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fe93ab9c-ae4e-48af-a423-74e32a601101" volumeName="kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018568    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61f54971-c57c-405c-93a8-19fcc3e50908" volumeName="kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018581    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="867d79d0-81fd-4c82-9a3a-6d926b0a0ae4" volumeName="kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018593    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1839b99-bbb7-4a20-9507-7d72dbbe354d" volumeName="kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018606    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" volumeName="kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018618    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d8081555-b5de-4ffa-bbe7-e2931b269310" volumeName="kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018630    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e0347d66-da9b-4703-85f2-cfe2f25eb1d9" volumeName="kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018643    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018654    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018667    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6466c095-0df6-4688-abc3-2f6f212471a0" volumeName="kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018681    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7bd53292-a0c6-41a8-8a39-41a27853cb73" volumeName="kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018692    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="867d79d0-81fd-4c82-9a3a-6d926b0a0ae4" volumeName="kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018705    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" volumeName="kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca" seLinuxMountContext=""
Oct 29 23:13:09.018731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018722    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e525b897-14cf-4e83-a016-60ab539f490d" volumeName="kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018744    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018757    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018770    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61f54971-c57c-405c-93a8-19fcc3e50908" volumeName="kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018782    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e5f497d2-8302-4707-b479-4dca872829e3" volumeName="kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018794    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018807    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f63a0452-9661-4fbb-9396-9ac7b5524dd3" volumeName="kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018819    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018830    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="867d79d0-81fd-4c82-9a3a-6d926b0a0ae4" volumeName="kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018842    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d42a6856-295d-4cf1-8a6e-691fcc073a34" volumeName="kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018855    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de6957c0-aaa9-449b-b5be-4f29212f40ce" volumeName="kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018868    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018880    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018893    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="86fa6768-086e-4434-9d30-cc860a28acc6" volumeName="kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018905    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018918    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37f1ad0a-20e9-45bb-9340-37f5dc7410d3" volumeName="kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018930    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018943    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="61f54971-c57c-405c-93a8-19fcc3e50908" volumeName="kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018955    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5c0147f-93d0-431f-a294-93c26838eff3" volumeName="kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018967    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.018996    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22482cc8-beda-4302-b88e-fed36f1bb7db" volumeName="kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019008    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e" volumeName="kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019021    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019034    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e1cae4c-99a8-4eb9-a957-1b38f9460e55" volumeName="kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019054    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3" volumeName="kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019067    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3" volumeName="kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019080    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3" volumeName="kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019115    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de6957c0-aaa9-449b-b5be-4f29212f40ce" volumeName="kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019137    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="adbe2362-88df-49bd-9692-e1311e28c396" volumeName="kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019150    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de4fcb6f-b5d4-4806-882a-8145a71c40d3" volumeName="kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019163    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de6957c0-aaa9-449b-b5be-4f29212f40ce" volumeName="kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019176    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e525b897-14cf-4e83-a016-60ab539f490d" volumeName="kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019188    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e82f1b60-106c-4927-ad20-18a9cdf92709" volumeName="kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019201    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="37f1ad0a-20e9-45bb-9340-37f5dc7410d3" volumeName="kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019214    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="867d79d0-81fd-4c82-9a3a-6d926b0a0ae4" volumeName="kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019227    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d42a6856-295d-4cf1-8a6e-691fcc073a34" volumeName="kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019241    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d42a6856-295d-4cf1-8a6e-691fcc073a34" volumeName="kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019254    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d7081746-11b4-4ea2-b641-2fc60782a2cf" volumeName="kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019267    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0641c627-1da8-47bc-bca1-04ade039e493" volumeName="kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019280    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019292    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1839b99-bbb7-4a20-9507-7d72dbbe354d" volumeName="kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019304    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" volumeName="kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019317    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de4fcb6f-b5d4-4806-882a-8145a71c40d3" volumeName="kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019328    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="de6957c0-aaa9-449b-b5be-4f29212f40ce" volumeName="kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019338    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fbaebde-1120-4288-a1bc-24d3f97bf85b" volumeName="kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019348    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019360    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1839b99-bbb7-4a20-9507-7d72dbbe354d" volumeName="kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019371    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae915ed-0747-410a-acec-a18068a98efe" volumeName="kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019383    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" volumeName="kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019395    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" volumeName="kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019407    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6b5a784-3628-4301-a7c5-f193012659a0" volumeName="kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019425    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7" volumeName="kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019437    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e1cae4c-99a8-4eb9-a957-1b38f9460e55" volumeName="kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019448    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="867d79d0-81fd-4c82-9a3a-6d926b0a0ae4" volumeName="kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019459    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aae915ed-0747-410a-acec-a18068a98efe" volumeName="kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019489    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e29dccc7-5622-4583-b74e-220df6ff5a34" volumeName="kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019503    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47425113-93db-453d-ac5e-b1b778b9bfe3" volumeName="kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019515    5111 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" volumeName="kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca" seLinuxMountContext=""
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019527    5111 reconstruct.go:97] "Volume reconstruction finished"
Oct 29 23:13:09.019607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.019534    5111 reconciler.go:29] "Reconciler: start to sync state"
Oct 29 23:13:09.022060 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022040    5111 factory.go:55] Registering systemd factory
Oct 29 23:13:09.022164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022153    5111 factory.go:223] Registration of the systemd container factory successfully
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022654    5111 factory.go:153] Registering CRI-O factory
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022672    5111 factory.go:223] Registration of the crio container factory successfully
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022742    5111 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022776    5111 factory.go:103] Registering Raw factory
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.022793    5111 manager.go:1199] Started watching for new ooms in manager
Oct 29 23:13:09.023248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.023228    5111 manager.go:319] Starting recovery of all containers
Oct 29 23:13:09.040192 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.040170    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:09.040286 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.040180    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:09.044847 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.044823    5111 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 29 23:13:09.053937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.053914    5111 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 29 23:13:09.053937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.053936    5111 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 29 23:13:09.054122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.053955    5111 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 29 23:13:09.054122 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.054051    5111 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 29 23:13:09.061114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.061094    5111 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:09.061204 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.061099    5111 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 29 23:13:09.062365 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.062329    5111 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:09.069627 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.069609    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:09.069723 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.069638    5111 generic.go:356] "Generic (PLEG): container finished" podID="c765f10e-91f1-4c9f-9ffd-2740b2bde8b9" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2" exitCode=255
Oct 29 23:13:09.074846 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.074811    5111 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:09.076850 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.076282    5111 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 29 23:13:09.095391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095369    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="80c2a155a024f0922b41624e391b8f1ac951e167b01fb04f781573a741b30e5a" exitCode=0
Oct 29 23:13:09.095391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095389    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="b92fcf40dee55a08ba4ce58719dd49671bb35bd6ebd43f5697dbfa869fa65566" exitCode=0
Oct 29 23:13:09.095511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095396    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="d0441319ac506a6cb48c26b019de02a36e2e3d31d12f4ea959adf8d13445fc40" exitCode=0
Oct 29 23:13:09.095511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095403    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="5ea72a85f3a8e6a37b2f713d81864b82a38dad67e46cefdee41804624143b313" exitCode=0
Oct 29 23:13:09.095511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095408    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="7a5da1493cbdc3d4a14570059707f32239e8aa780da3b22b5afb89053c6e53b9" exitCode=0
Oct 29 23:13:09.095511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.095412    5111 generic.go:356] "Generic (PLEG): container finished" podID="61f54971-c57c-405c-93a8-19fcc3e50908" containerID="76c4881882da6dd5d3670db0cd12c2ed850d874e46a8a7d08e8e347962d56c2d" exitCode=0
Oct 29 23:13:09.097148 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.097125    5111 generic.go:356] "Generic (PLEG): container finished" podID="8ad8bdceb419b718ac0187fcaf3ee427" containerID="69bc079f1819c4bf11be9d2e1b3d679b5c15b16d7f08c9ca1077f9353ce33f9c" exitCode=0
Oct 29 23:13:09.118822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.118804    5111 manager.go:324] Recovery completed
Oct 29 23:13:09.138265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138231    5111 cpu_manager.go:226] "Starting" policy="none"
Oct 29 23:13:09.138265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138265    5111 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 29 23:13:09.138422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138285    5111 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 29 23:13:09.138494 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138474    5111 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 29 23:13:09.138587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138560    5111 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 29 23:13:09.138638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138591    5111 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 29 23:13:09.138638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138600    5111 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 29 23:13:09.138638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138609    5111 policy_none.go:50] "Start"
Oct 29 23:13:09.138638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138630    5111 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 29 23:13:09.138820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138645    5111 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 29 23:13:09.138904 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138889    5111 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 29 23:13:09.138955 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138908    5111 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 29 23:13:09.138955 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.138917    5111 policy_none.go:44] "Start"
Oct 29 23:13:09.144044 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.144019    5111 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 29 23:13:09.147623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.147605    5111 manager.go:341] "Starting Device Plugin manager"
Oct 29 23:13:09.147706 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.147635    5111 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 29 23:13:09.147706 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.147646    5111 server.go:87] "Starting device plugin registration server"
Oct 29 23:13:09.147920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.147910    5111 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 29 23:13:09.147958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.147921    5111 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 29 23:13:09.148049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.148034    5111 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 29 23:13:09.148187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.148152    5111 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 29 23:13:09.148253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.148188    5111 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 29 23:13:09.148253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.148195    5111 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 29 23:13:09.148336 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.148247    5111 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:13:09.148176926Z","UUID":"45c0aac5-a635-4374-8685-f8f51a54f45c","Handler":null,"Name":"","Endpoint":""}
Oct 29 23:13:09.149447 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.149429    5111 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 29 23:13:09.149559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.149537    5111 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 29 23:13:09.149649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.149574    5111 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 29 23:13:09.154175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154134    5111 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"]
Oct 29 23:13:09.154829 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154793    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal" event={"ID":"9bc940380ed911d9e6c421c48f4b1a6a","Type":"ContainerStarted","Data":"1ad9887bb3a6be48e60ec7dc3f7fe1c94cb638fe980145e381b9f25b28299cc9"}
Oct 29 23:13:09.154829 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154828    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal" event={"ID":"9bc940380ed911d9e6c421c48f4b1a6a","Type":"ContainerStarted","Data":"04b3fc61e60e2207976551817328ce867ef387f4f28026ba8cddbbb2bfa5f5ef"}
Oct 29 23:13:09.154961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154899    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerStarted","Data":"0d632da8f4d6ccf2330cdf0f8ed709815ace5f8fe85b1384bba9ad6e0173e4ca"}
Oct 29 23:13:09.154961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154829    5111 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.154961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154915    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerDied","Data":"69bc079f1819c4bf11be9d2e1b3d679b5c15b16d7f08c9ca1077f9353ce33f9c"}
Oct 29 23:13:09.154961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.154933    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal" event={"ID":"8ad8bdceb419b718ac0187fcaf3ee427","Type":"ContainerStarted","Data":"3591dd0bace545c7e710b239e4e6fca37f7ed0f75d5028bb59954309b6e726cc"}
Oct 29 23:13:09.155151 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.155067    5111 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.178391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.178375    5111 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:09.178461 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.178410    5111 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.181069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.181050    5111 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:09.181120 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:09.181092    5111 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.248285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.248236    5111 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 29 23:13:09.249243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.249228    5111 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientMemory"
Oct 29 23:13:09.249326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.249256    5111 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasNoDiskPressure"
Oct 29 23:13:09.249326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.249271    5111 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-12.ec2.internal" event="NodeHasSufficientPID"
Oct 29 23:13:09.249633 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.249620    5111 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.261537 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.261519    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.261612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.261581    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.261654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.261617    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.281368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.281347    5111 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.281432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.281392    5111 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362191    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362142    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362245    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362266    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362293    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9bc940380ed911d9e6c421c48f4b1a6a-config\") pod \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" (UID: \"9bc940380ed911d9e6c421c48f4b1a6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:09.362422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:09.362341    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ad8bdceb419b718ac0187fcaf3ee427-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" (UID: \"8ad8bdceb419b718ac0187fcaf3ee427\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:10.003927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.003899    5111 apiserver.go:52] "Watching apiserver"
Oct 29 23:13:10.056686 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.056661    5111 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 29 23:13:10.057349 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.057322    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-7g7tb","openshift-network-diagnostics/network-check-target-q9xkk","openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8","openshift-insights/insights-runtime-extractor-mcxnv","kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal","openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw","kube-system/global-pull-secret-syncer-64m88","openshift-image-registry/image-registry-78868f4995-5rhvn","openshift-image-registry/image-registry-54576fc98-lv7kk","kube-system/konnectivity-agent-b9h9g","openshift-dns/node-resolver-4blvf","openshift-network-console/networking-console-plugin-86c478bb88-n72kb","openshift-network-operator/iptables-alerter-nwtsk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk","openshift-ovn-kubernetes/ovnkube-node-jdnh8","openshift-insights/insights-operator-7c4986d449-n4774","kube-system/kas-connection-checker-685895c7c9-hhgrs","openshift-multus/network-metrics-daemon-vqlgv","kube-system/kas-connection-checker-685895c7c9-c68qr","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp","openshift-multus/multus-l9kgz","openshift-console-operator/console-operator-7cd56b8dbc-7v65t","openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh","openshift-ingress/router-default-cddcfffcc-8wmcx","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn","openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t","openshift-cluster-node-tuning-operator/tuned-lhps4","openshift-insights/periodic-gathering-fth5d-rckp7","openshift-service-ca/service-ca-5fffc97bdf-jhgs7","openshift-ingress-canary/ingress-canary-hvb2t","openshift-image-registry/node-ca-g9m5t","kube-system/kas-connection-checker-685895c7c9-88q8b","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst","openshift-dns/dns-default-f8xtp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal","openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"]
Oct 29 23:13:10.058493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.058477    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.058812 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.058790    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.059945 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.059927    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.060901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.060886    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.062043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.062025    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.063505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.063491    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.069683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.069662    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.070778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.070760    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.073247 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.073228    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.074435 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.074418    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.074560 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.074544    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.074849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.074832    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Oct 29 23:13:10.075017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075003    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.075100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075086    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Oct 29 23:13:10.075154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075137    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.075154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075150    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Oct 29 23:13:10.075298 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075287    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-gb2nj\""
Oct 29 23:13:10.075486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.075472    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Oct 29 23:13:10.076278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.076265    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.077380 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077364    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.077493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077450    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.077493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077476    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 29 23:13:10.077588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077577    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.077630 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077586    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.077693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077681    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Oct 29 23:13:10.077794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077782    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 29 23:13:10.077998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.077988    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Oct 29 23:13:10.078128 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.078107    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hfr96\""
Oct 29 23:13:10.078128 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.078111    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.078274 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.078208    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2p5c8\""
Oct 29 23:13:10.078307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.078280    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l8rj6\""
Oct 29 23:13:10.079171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.079121    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:10.087461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.087445    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Oct 29 23:13:10.094989 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.094955    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 29 23:13:10.095405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095386    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 29 23:13:10.095512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095423    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 29 23:13:10.095512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095424    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-pphcq\""
Oct 29 23:13:10.095512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095442    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qbkzf\""
Oct 29 23:13:10.095512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095485    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tldxj\""
Oct 29 23:13:10.095694 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095564    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 29 23:13:10.095694 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095591    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 29 23:13:10.095694 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095629    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-phkn8\""
Oct 29 23:13:10.095856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095826    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-6nt2h\""
Oct 29 23:13:10.095893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095858    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.096008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095967    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Oct 29 23:13:10.096008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095991    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7zkzr\""
Oct 29 23:13:10.096008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.095989    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-89z4k\""
Oct 29 23:13:10.096159 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096022    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h9zs2\""
Oct 29 23:13:10.096159 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096154    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-t8tv9\""
Oct 29 23:13:10.096252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096172    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Oct 29 23:13:10.096252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096212    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ss48m\""
Oct 29 23:13:10.096252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096232    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8km5l\""
Oct 29 23:13:10.096354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096280    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-g2zrz\""
Oct 29 23:13:10.096354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096337    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sx9m6\""
Oct 29 23:13:10.096354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096339    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-pgvzg\""
Oct 29 23:13:10.096474 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096354    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Oct 29 23:13:10.096474 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096415    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 29 23:13:10.096474 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096440    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 29 23:13:10.096614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096600    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.096755 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096720    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 29 23:13:10.096837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096755    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Oct 29 23:13:10.096900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096850    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 29 23:13:10.096900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.096892    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.097389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097365    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.097477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097408    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.097477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097440    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hwcbb\""
Oct 29 23:13:10.097477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097471    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097487    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097498    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097499    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097524    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097529    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097551    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097570    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097592    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097577    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097607    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097621    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.097634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097636    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097650    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097728    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qprkx\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097722    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097797    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-hn6wx\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097836    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097849    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097855    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097870    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097890    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.097823    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098054    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098055    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098067    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098083    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098143    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Oct 29 23:13:10.098197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098173    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098219    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098265    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-jb6zf\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098268    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098339    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pnrbh\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098370    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098380    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098391    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.098747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098390    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098799    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098854    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098910    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9w58g\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098920    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098944    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.098996    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099021    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099022    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099028    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099046    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099067    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 29 23:13:10.099094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099096    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 29 23:13:10.099565 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099123    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-gj5kk\""
Oct 29 23:13:10.099565 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099147    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 29 23:13:10.099668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099649    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 29 23:13:10.099718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099670    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2dl5f\""
Oct 29 23:13:10.099718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.099687    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 29 23:13:10.107869 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.107845    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 29 23:13:10.107965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.107944    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2vcdm\""
Oct 29 23:13:10.108722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.108703    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Oct 29 23:13:10.109321 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.109308    5111 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:10.109431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.109419    5111 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:10.117268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.117250    5111 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 29 23:13:10.144296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.144273    5111 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:10.144384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.144304    5111 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 29 23:13:10.144384 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.144330    5111 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:10.144384 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.144350    5111 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-12.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-12.ec2.internal"
Oct 29 23:13:10.165227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165202    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:10.165317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165235    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.165317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165253    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.165317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165269    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.165317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165287    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2db6t\" (UniqueName: \"kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.165317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165308    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.165556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165367    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwlfl\" (UniqueName: \"kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.165556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165452    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.165556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165486    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.165556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165533    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.165734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165581    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.165734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165623    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.165734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165654    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.165734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165670    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.165895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165830    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.165895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165878    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.166012 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.165998    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.166075 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166055    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g259q\" (UniqueName: \"kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.166127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166083    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-snapshots\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.166127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166117    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss25j\" (UniqueName: \"kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166128    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166146    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166172    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166190    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d8081555-b5de-4ffa-bbe7-e2931b269310-serviceca\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166196    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.166234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166223    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqllh\" (UniqueName: \"kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166246    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166280    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166305    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166329    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166335    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-tmp\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166360    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166388    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfnd\" (UniqueName: \"kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd\") pod \"kas-connection-checker-685895c7c9-c68qr\" (UID: \"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7\") " pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166414    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166412    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166475    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-env-overrides\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.166559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166540    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166572    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166595    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e29dccc7-5622-4583-b74e-220df6ff5a34-original-pull-secret\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166579    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166619    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166649    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166671    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k4nt\" (UniqueName: \"kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt\") pod \"volume-data-source-validator-58c9f5f46d-g2czn\" (UID: \"f5c0147f-93d0-431f-a294-93c26838eff3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166712    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166742    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166746    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-stats-auth\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166772    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166862    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166897    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-default-certificate\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166920    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166950    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.166997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.166997    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvs8h\" (UniqueName: \"kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167049    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167082    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167107    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167169    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167202    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167233    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167172    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167262    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167271    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167346    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167384    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167421    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167451    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvxnl\" (UniqueName: \"kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167478    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167509    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167514    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-konnectivity-ca\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167524    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167481    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cni-binary-copy\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167573    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.167587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167598    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167616    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167623    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167661    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167669    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/22482cc8-beda-4302-b88e-fed36f1bb7db-data-volume\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167707    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167749    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxwxd\" (UniqueName: \"kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167760    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167780    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz42q\" (UniqueName: \"kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167796    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-cabundle\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167811    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167847    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167875    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167911    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.167961    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168034    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6mc\" (UniqueName: \"kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168063    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5f497d2-8302-4707-b479-4dca872829e3-tmp-dir\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168088    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168106    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whdld\" (UniqueName: \"kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld\") pod \"kas-connection-checker-685895c7c9-hhgrs\" (UID: \"7bd53292-a0c6-41a8-8a39-41a27853cb73\") " pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168150    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168253    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.168327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168290    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x2bw\" (UniqueName: \"kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw\") pod \"migrator-66df5d496b-9ft4t\" (UID: \"e0347d66-da9b-4703-85f2-cfe2f25eb1d9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168324    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168382    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9grgn\" (UniqueName: \"kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168410    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75xdt\" (UniqueName: \"kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168443    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168499    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168512    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168536    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168554    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168569    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168589    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168613    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168637    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmdn8\" (UniqueName: \"kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168663    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168693    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168721    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168768    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-config\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168766    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168795    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/61f54971-c57c-405c-93a8-19fcc3e50908-cni-binary-copy\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168819    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168931    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.168951    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-daemon-config\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169035    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169068    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-trusted-ca\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169069    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169121    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169157    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bkm9\" (UniqueName: \"kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169170    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42a6856-295d-4cf1-8a6e-691fcc073a34-tmp-dir\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169184    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/aae915ed-0747-410a-acec-a18068a98efe-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.169223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169235    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-serving-cert\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169266    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169300    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169326    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169415    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169441    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d42a6856-295d-4cf1-8a6e-691fcc073a34-config-volume\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169460    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169493    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2b6r\" (UniqueName: \"kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169556    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169561    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169647    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169695    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mn8r\" (UniqueName: \"kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r\") pod \"kas-connection-checker-685895c7c9-88q8b\" (UID: \"86fa6768-086e-4434-9d30-cc860a28acc6\") " pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169846    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169873    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169909    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169938    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.169961    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170040    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170069    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170093    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170111    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a1839b99-bbb7-4a20-9507-7d72dbbe354d-signing-key\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170118    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170144    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170167    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170177    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170190    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170232    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe93ab9c-ae4e-48af-a423-74e32a601101-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170244    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-service-ca-bundle\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170260    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/7e1cae4c-99a8-4eb9-a957-1b38f9460e55-agent-certs\") pod \"konnectivity-agent-b9h9g\" (UID: \"7e1cae4c-99a8-4eb9-a957-1b38f9460e55\") " pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170251    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwp8j\" (UniqueName: \"kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j\") pod \"network-check-source-6f78d59f99-6djdh\" (UID: \"0641c627-1da8-47bc-bca1-04ade039e493\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170299    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170316    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/47425113-93db-453d-ac5e-b1b778b9bfe3-nginx-conf\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170327    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170368    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170401    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dnjr\" (UniqueName: \"kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.170405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170428    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovnkube-script-lib\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170428    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjv9k\" (UniqueName: \"kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170466    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl9zc\" (UniqueName: \"kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170493    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170498    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e525b897-14cf-4e83-a016-60ab539f490d-config\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170517    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170540    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170564    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170590    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170619    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170652    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170680    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170685    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-tmp\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170738    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170767    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170822    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170826    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-config\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170873    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170902    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170912    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-serving-cert\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.170928    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171094    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171166    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171200    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171218    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f63a0452-9661-4fbb-9396-9ac7b5524dd3-cert\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171299    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171320    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adbe2362-88df-49bd-9692-e1311e28c396-iptables-alerter-script\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171336    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171333    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fe93ab9c-ae4e-48af-a423-74e32a601101-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171385    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e525b897-14cf-4e83-a016-60ab539f490d-serving-cert\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171457    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171494    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171527    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171569    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsbhp\" (UniqueName: \"kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp\") pod \"migrator-66df5d496b-sz7lb\" (UID: \"e82f1b60-106c-4927-ad20-18a9cdf92709\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171604    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171652    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-tuned\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171652    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171685    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.171699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171712    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171729    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171762    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171795    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89dfq\" (UniqueName: \"kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171821    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rpww\" (UniqueName: \"kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171842    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171858    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171874    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.171899    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.172767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.172414    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.188722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.188701    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm68z\" (UniqueName: \"kubernetes.io/projected/4fbaebde-1120-4288-a1bc-24d3f97bf85b-kube-api-access-zm68z\") pod \"network-check-target-q9xkk\" (UID: \"4fbaebde-1120-4288-a1bc-24d3f97bf85b\") " pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:10.190511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.190494    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwlfl\" (UniqueName: \"kubernetes.io/projected/6466c095-0df6-4688-abc3-2f6f212471a0-kube-api-access-rwlfl\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.191056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.191037    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g259q\" (UniqueName: \"kubernetes.io/projected/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-kube-api-access-g259q\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.191593 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.191577    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2db6t\" (UniqueName: \"kubernetes.io/projected/fe93ab9c-ae4e-48af-a423-74e32a601101-kube-api-access-2db6t\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-bpnst\" (UID: \"fe93ab9c-ae4e-48af-a423-74e32a601101\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst"
Oct 29 23:13:10.201721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.201705    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75xdt\" (UniqueName: \"kubernetes.io/projected/a1839b99-bbb7-4a20-9507-7d72dbbe354d-kube-api-access-75xdt\") pod \"service-ca-5fffc97bdf-jhgs7\" (UID: \"a1839b99-bbb7-4a20-9507-7d72dbbe354d\") " pod="openshift-service-ca/service-ca-5fffc97bdf-jhgs7"
Oct 29 23:13:10.201788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.201727    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl9zc\" (UniqueName: \"kubernetes.io/projected/c765f10e-91f1-4c9f-9ffd-2740b2bde8b9-kube-api-access-cl9zc\") pod \"console-operator-7cd56b8dbc-7v65t\" (UID: \"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:10.202138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.202123    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.203091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.203075    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6mc\" (UniqueName: \"kubernetes.io/projected/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-kube-api-access-7g6mc\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.203520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.203505    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.203574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.203522    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whdld\" (UniqueName: \"kubernetes.io/projected/7bd53292-a0c6-41a8-8a39-41a27853cb73-kube-api-access-whdld\") pod \"kas-connection-checker-685895c7c9-hhgrs\" (UID: \"7bd53292-a0c6-41a8-8a39-41a27853cb73\") " pod="kube-system/kas-connection-checker-685895c7c9-hhgrs"
Oct 29 23:13:10.204048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.204033    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mn8r\" (UniqueName: \"kubernetes.io/projected/86fa6768-086e-4434-9d30-cc860a28acc6-kube-api-access-7mn8r\") pod \"kas-connection-checker-685895c7c9-88q8b\" (UID: \"86fa6768-086e-4434-9d30-cc860a28acc6\") " pod="kube-system/kas-connection-checker-685895c7c9-88q8b"
Oct 29 23:13:10.204426 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.204411    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bkm9\" (UniqueName: \"kubernetes.io/projected/d8081555-b5de-4ffa-bbe7-e2931b269310-kube-api-access-6bkm9\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.204557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.204537    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89dfq\" (UniqueName: \"kubernetes.io/projected/f63a0452-9661-4fbb-9396-9ac7b5524dd3-kube-api-access-89dfq\") pod \"ingress-canary-hvb2t\" (UID: \"f63a0452-9661-4fbb-9396-9ac7b5524dd3\") " pod="openshift-ingress-canary/ingress-canary-hvb2t"
Oct 29 23:13:10.204623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.204608    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqllh\" (UniqueName: \"kubernetes.io/projected/d42a6856-295d-4cf1-8a6e-691fcc073a34-kube-api-access-zqllh\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.205299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.205284    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz42q\" (UniqueName: \"kubernetes.io/projected/e5f497d2-8302-4707-b479-4dca872829e3-kube-api-access-nz42q\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.205806 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.205790    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxwxd\" (UniqueName: \"kubernetes.io/projected/aae915ed-0747-410a-acec-a18068a98efe-kube-api-access-rxwxd\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.207533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.207480    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjv9k\" (UniqueName: \"kubernetes.io/projected/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-kube-api-access-hjv9k\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.207871 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.207855    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsbhp\" (UniqueName: \"kubernetes.io/projected/e82f1b60-106c-4927-ad20-18a9cdf92709-kube-api-access-gsbhp\") pod \"migrator-66df5d496b-sz7lb\" (UID: \"e82f1b60-106c-4927-ad20-18a9cdf92709\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-sz7lb"
Oct 29 23:13:10.210149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.210134    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvxnl\" (UniqueName: \"kubernetes.io/projected/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-api-access-wvxnl\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.210259 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.210244    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x2bw\" (UniqueName: \"kubernetes.io/projected/e0347d66-da9b-4703-85f2-cfe2f25eb1d9-kube-api-access-5x2bw\") pod \"migrator-66df5d496b-9ft4t\" (UID: \"e0347d66-da9b-4703-85f2-cfe2f25eb1d9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-9ft4t"
Oct 29 23:13:10.211513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.211484    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k4nt\" (UniqueName: \"kubernetes.io/projected/f5c0147f-93d0-431f-a294-93c26838eff3-kube-api-access-8k4nt\") pod \"volume-data-source-validator-58c9f5f46d-g2czn\" (UID: \"f5c0147f-93d0-431f-a294-93c26838eff3\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-g2czn"
Oct 29 23:13:10.211634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.211617    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmdn8\" (UniqueName: \"kubernetes.io/projected/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-kube-api-access-mmdn8\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.211696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.211632    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.211696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.211656    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfnd\" (UniqueName: \"kubernetes.io/projected/615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7-kube-api-access-6mfnd\") pod \"kas-connection-checker-685895c7c9-c68qr\" (UID: \"615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7\") " pod="kube-system/kas-connection-checker-685895c7c9-c68qr"
Oct 29 23:13:10.212087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.212068    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2b6r\" (UniqueName: \"kubernetes.io/projected/e61d9ca2-dabd-4378-bee7-ecacbfd366a6-kube-api-access-f2b6r\") pod \"insights-operator-7c4986d449-n4774\" (UID: \"e61d9ca2-dabd-4378-bee7-ecacbfd366a6\") " pod="openshift-insights/insights-operator-7c4986d449-n4774"
Oct 29 23:13:10.212419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.212404    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9grgn\" (UniqueName: \"kubernetes.io/projected/adbe2362-88df-49bd-9692-e1311e28c396-kube-api-access-9grgn\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.213599 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.213580    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rpww\" (UniqueName: \"kubernetes.io/projected/e525b897-14cf-4e83-a016-60ab539f490d-kube-api-access-7rpww\") pod \"service-ca-operator-544d749c8c-nggm8\" (UID: \"e525b897-14cf-4e83-a016-60ab539f490d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8"
Oct 29 23:13:10.213895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.213874    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"periodic-gathering-fth5d-rckp7\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") " pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:10.213928 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.213907    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss25j\" (UniqueName: \"kubernetes.io/projected/61f54971-c57c-405c-93a8-19fcc3e50908-kube-api-access-ss25j\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.214698 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.214682    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwp8j\" (UniqueName: \"kubernetes.io/projected/0641c627-1da8-47bc-bca1-04ade039e493-kube-api-access-qwp8j\") pod \"network-check-source-6f78d59f99-6djdh\" (UID: \"0641c627-1da8-47bc-bca1-04ade039e493\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-6djdh"
Oct 29 23:13:10.215280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.215265    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.215943 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.215928    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dnjr\" (UniqueName: \"kubernetes.io/projected/d7081746-11b4-4ea2-b641-2fc60782a2cf-kube-api-access-8dnjr\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.216437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.216417    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvs8h\" (UniqueName: \"kubernetes.io/projected/de4fcb6f-b5d4-4806-882a-8145a71c40d3-kube-api-access-cvs8h\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.272506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272454    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.272506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272489    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272513    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272537    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272574    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272579    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-netns\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272591    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-socket-dir-parent\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272596    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.272604    5111 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:10.272656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272651    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-etc-kubernetes\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.272667    5111 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272628    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272671    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-k8s-cni-cncf-io\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272677    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-sys-fs\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.272702    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.772682466 +0000 UTC m=+1.858965601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272725    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272761    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.272770    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.772752618 +0000 UTC m=+1.859035733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : secret "metrics-daemon-secret" not found
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272791    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272815    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272829    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272845    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272848    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-kubernetes\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272872    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272874    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-netns\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272888    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-var-lib-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272895    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272896    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/22482cc8-beda-4302-b88e-fed36f1bb7db-crio-socket\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272919    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272923    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.272949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272953    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-dbus\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.272950    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273006    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273031    5111 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273066    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273031    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-var-lib-kubelet\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273071    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.773058538 +0000 UTC m=+1.859341674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273093    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-run-multus-certs\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273111    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273136    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273160    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273167    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-netd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273168    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-system-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273181    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273192    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adbe2362-88df-49bd-9692-e1311e28c396-host-slash\") pod \"iptables-alerter-nwtsk\" (UID: \"adbe2362-88df-49bd-9692-e1311e28c396\") " pod="openshift-network-operator/iptables-alerter-nwtsk"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273213    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273226    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-multus\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273264    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-systemd\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273264    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273275    5111 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273310    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273336    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273351    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e29dccc7-5622-4583-b74e-220df6ff5a34-kubelet-config\") pod \"global-pull-secret-syncer-64m88\" (UID: \"e29dccc7-5622-4583-b74e-220df6ff5a34\") " pod="kube-system/global-pull-secret-syncer-64m88"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273354    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-slash\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273359    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273395    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273396    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-node-log\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273404    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-cni-bin\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273411    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273437    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273443    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-cni-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273442    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-system-cni-dir\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273461    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273479    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273482    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-kubelet\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273494    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273510    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273529    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273551    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273555    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-hostroot\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273562    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-kubelet\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273575    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273579    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-systemd-units\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273581    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-cnibin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273587    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/61f54971-c57c-405c-93a8-19fcc3e50908-cnibin\") pod \"multus-additional-cni-plugins-7g7tb\" (UID: \"61f54971-c57c-405c-93a8-19fcc3e50908\") " pod="openshift-multus/multus-additional-cni-plugins-7g7tb"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273599    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273613    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273604    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-log-socket\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273641    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273667    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273691    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273725    5111 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273757    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d8081555-b5de-4ffa-bbe7-e2931b269310-host\") pod \"node-ca-g9m5t\" (UID: \"d8081555-b5de-4ffa-bbe7-e2931b269310\") " pod="openshift-image-registry/node-ca-g9m5t"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273764    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273699    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-conf\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273757    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-run\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273780    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.773764902 +0000 UTC m=+1.860048033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273720    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-registration-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.273784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273807    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273833    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273857    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273879    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273885    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysconfig\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273898    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273901    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273911    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273921    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-systemd\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.273954    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.273966    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.773952305 +0000 UTC m=+1.860235449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274009    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-multus-conf-dir\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274020    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274029    5111 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274046    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274049    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-ovn\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274076    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.774063512 +0000 UTC m=+1.860346638 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274090    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274095    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-lib-modules\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274107    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274098    5111 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274115    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3-host-var-lib-cni-bin\") pod \"multus-l9kgz\" (UID: \"9ec7f47d-ee1b-4f87-8b0b-5aa9b354f2f3\") " pod="openshift-multus/multus-l9kgz"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274124    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274147    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274172    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5f497d2-8302-4707-b479-4dca872829e3-hosts-file\") pod \"node-resolver-4blvf\" (UID: \"e5f497d2-8302-4707-b479-4dca872829e3\") " pod="openshift-dns/node-resolver-4blvf"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274188    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274213    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.774199137 +0000 UTC m=+1.860482277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274211    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-etc-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274216    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-device-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274233    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274259    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274297    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274302    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-sysctl-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274303    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.774291676 +0000 UTC m=+1.860574811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274335    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-host\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274339    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274349    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274370    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-run-openvswitch\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274380    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274405    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274454    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-etc-modprobe-d\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274429    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274475    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de4fcb6f-b5d4-4806-882a-8145a71c40d3-sys\") pod \"tuned-lhps4\" (UID: \"de4fcb6f-b5d4-4806-882a-8145a71c40d3\") " pod="openshift-cluster-node-tuning-operator/tuned-lhps4"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274504    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274522    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274534    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274544    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274551    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/867d79d0-81fd-4c82-9a3a-6d926b0a0ae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jdnh8\" (UID: \"867d79d0-81fd-4c82-9a3a-6d926b0a0ae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274560    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.274576    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:10.774561676 +0000 UTC m=+1.860844796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.274618    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6466c095-0df6-4688-abc3-2f6f212471a0-socket-dir\") pod \"aws-ebs-csi-driver-node-447tk\" (UID: \"6466c095-0df6-4688-abc3-2f6f212471a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-447tk"
Oct 29 23:13:10.276260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.275748    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/37f1ad0a-20e9-45bb-9340-37f5dc7410d3-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-q47dp\" (UID: \"37f1ad0a-20e9-45bb-9340-37f5dc7410d3\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.278688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.276687    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d42a6856-295d-4cf1-8a6e-691fcc073a34-metrics-tls\") pod \"dns-default-f8xtp\" (UID: \"d42a6856-295d-4cf1-8a6e-691fcc073a34\") " pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.312108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.308788    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:10.361758 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.361736    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"
Oct 29 23:13:10.361863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.361812    5111 scope.go:122] "RemoveContainer" containerID="2ce0f94a5d1e9af70a9b0dd4cf8632d5808344198fb9bb5aefecbeb7c8a9c8d2"
Oct 29 23:13:10.371080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.371056    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:10.483304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.483232    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:10.647454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.647416    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:10.661310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.661292    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp"]
Oct 29 23:13:10.674599 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.673396    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f8xtp"]
Oct 29 23:13:10.677389 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:10.677353    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42a6856_295d_4cf1_8a6e_691fcc073a34.slice/crio-7380b95f736a66cf7eec306d6f3d4f8dbf7114f20a1b7eb5ea3e5ff868cf411a WatchSource:0}: Error finding container 7380b95f736a66cf7eec306d6f3d4f8dbf7114f20a1b7eb5ea3e5ff868cf411a: Status 404 returned error can't find the container with id 7380b95f736a66cf7eec306d6f3d4f8dbf7114f20a1b7eb5ea3e5ff868cf411a
Oct 29 23:13:10.679854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.679837    5111 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:13:10.779575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779550    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:10.779657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779596    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:10.779657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779620    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:10.779730 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779692    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:10.779730 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779695    5111 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:10.779730 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779722    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.779813 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779747    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.779733791 +0000 UTC m=+2.866016922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:10.779813 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779702    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:13:10.779813 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779787    5111 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:10.779813 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779775    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.779813 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779692    5111 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779817    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.779802756 +0000 UTC m=+2.866085868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779836    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.779827261 +0000 UTC m=+2.866110379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779848    5111 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779851    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.779844444 +0000 UTC m=+2.866127556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.779904    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.77989241 +0000 UTC m=+2.866175522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : configmap "kube-rbac-proxy" not found
Oct 29 23:13:10.779994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.779939    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.780006    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.780042    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:10.780072    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780110    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780118    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.78010388 +0000 UTC m=+2.866386994 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780120    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780134    5111 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780158    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.780148637 +0000 UTC m=+2.866431753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:13:10.780194 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780172    5111 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 29 23:13:10.780444 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780175    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.780163091 +0000 UTC m=+2.866446220 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:10.780444 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:10.780222    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs podName:303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e nodeName:}" failed. No retries permitted until 2026-10-29 23:13:11.780209581 +0000 UTC m=+2.866492693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs") pod "network-metrics-daemon-vqlgv" (UID: "303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e") : secret "metrics-daemon-secret" not found
Oct 29 23:13:11.091542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.091507    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:11.114135 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.114103    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8xtp" event={"ID":"d42a6856-295d-4cf1-8a6e-691fcc073a34","Type":"ContainerStarted","Data":"7380b95f736a66cf7eec306d6f3d4f8dbf7114f20a1b7eb5ea3e5ff868cf411a"}
Oct 29 23:13:11.116095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.116066    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp" event={"ID":"37f1ad0a-20e9-45bb-9340-37f5dc7410d3","Type":"ContainerStarted","Data":"ffbc6f00e01c39cffe9fb1e459c6073cad1c25006d4cf67d67e2ffdb14e755d3"}
Oct 29 23:13:11.118432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.118413    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:11.118554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.118522    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t" event={"ID":"c765f10e-91f1-4c9f-9ffd-2740b2bde8b9","Type":"ContainerStarted","Data":"fcb9c274e063a4943aec9383c95c78007f99433fabbdde77a397abdf6537167a"}
Oct 29 23:13:11.294787 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.294713    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:11.455135 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.455101    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:11.618103 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.618070    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:11.656179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.656145    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:11.746157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.745741    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:11.790626 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790591    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790648    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790675    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790698    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790725    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790748    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:11.790789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.790771    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:11.791197 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.790749    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:11.791197 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791192    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78868f4995-5rhvn: secret "image-registry-tls" not found
Oct 29 23:13:11.791197 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791195    5111 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:11.791435 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.791316    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/22482cc8-beda-4302-b88e-fed36f1bb7db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:11.791435 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791365    5111 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Oct 29 23:13:11.791435 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791378    5111 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54576fc98-lv7kk: secret "image-registry-tls" not found
Oct 29 23:13:11.791435 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.790841    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.790817933 +0000 UTC m=+4.877101058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:11.791435 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791161    5111 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.791457    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791478    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791459457 +0000 UTC m=+4.877742586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791507    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791496316 +0000 UTC m=+4.877779432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791530    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls podName:52c5a942-07fa-4c10-8dd0-85bf3e9226f5 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791521339 +0000 UTC m=+4.877804457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls") pod "image-registry-78868f4995-5rhvn" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5") : secret "image-registry-tls" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791547    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls podName:f6b5a784-3628-4301-a7c5-f193012659a0 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791538641 +0000 UTC m=+4.877821758 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls") pod "image-registry-54576fc98-lv7kk" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0") : secret "image-registry-tls" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791550    5111 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791595    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls podName:aae915ed-0747-410a-acec-a18068a98efe nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791580809 +0000 UTC m=+4.877863922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-9wdnw" (UID: "aae915ed-0747-410a-acec-a18068a98efe") : secret "cluster-monitoring-operator-tls" not found
Oct 29 23:13:11.791709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.791634    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:11.792177 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791738    5111 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:11.792177 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:11.791773    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:13.791763152 +0000 UTC m=+4.878046264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:11.793364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.793341    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e-metrics-certs\") pod \"network-metrics-daemon-vqlgv\" (UID: \"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e\") " pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:11.818292 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.818264    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:11.864199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.864176    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlgv"
Oct 29 23:13:11.961003 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:11.959661    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:12.096196 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.096153    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:12.124065 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.123997    5111 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:12.166124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.166096    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqlgv"]
Oct 29 23:13:12.170372 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:12.170345    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod303a5fa2_fea5_4e46_91d2_f8b8ae3ce57e.slice/crio-74883e7a928bbc45067d826529fa3681711ad3887ae20c87a6da5ed63f8a7c24 WatchSource:0}: Error finding container 74883e7a928bbc45067d826529fa3681711ad3887ae20c87a6da5ed63f8a7c24: Status 404 returned error can't find the container with id 74883e7a928bbc45067d826529fa3681711ad3887ae20c87a6da5ed63f8a7c24
Oct 29 23:13:12.253861 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.253835    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:12.455533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.455470    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:12.652107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.652013    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:12.776105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.776077    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:12.776265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.776230    5111 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:12.781714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.781690    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-7v65t"
Oct 29 23:13:12.800380 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:12.800330    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:13.131274 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.131248    5111 generic.go:356] "Generic (PLEG): container finished" podID="e525b897-14cf-4e83-a016-60ab539f490d" containerID="b9e57c69414e3ed4408d6e7ffd55541e0934e13237bda6d6c5a4e764c21d541e" exitCode=0
Oct 29 23:13:13.131628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.131325    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8" event={"ID":"e525b897-14cf-4e83-a016-60ab539f490d","Type":"ContainerDied","Data":"b9e57c69414e3ed4408d6e7ffd55541e0934e13237bda6d6c5a4e764c21d541e"}
Oct 29 23:13:13.131710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.131690    5111 scope.go:122] "RemoveContainer" containerID="b9e57c69414e3ed4408d6e7ffd55541e0934e13237bda6d6c5a4e764c21d541e"
Oct 29 23:13:13.134463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.134393    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlgv" event={"ID":"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e","Type":"ContainerStarted","Data":"99d4ca25f665e9b3b23325397cad07f6bad019e8470a638879707ee97902e18d"}
Oct 29 23:13:13.134463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.134440    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlgv" event={"ID":"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e","Type":"ContainerStarted","Data":"74883e7a928bbc45067d826529fa3681711ad3887ae20c87a6da5ed63f8a7c24"}
Oct 29 23:13:13.136442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.136417    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8xtp" event={"ID":"d42a6856-295d-4cf1-8a6e-691fcc073a34","Type":"ContainerStarted","Data":"7d2e553481c3664f848ddfe8a52a08d7346bbb2e4c157bdce62dbb0676115284"}
Oct 29 23:13:13.136535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.136465    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f8xtp" event={"ID":"d42a6856-295d-4cf1-8a6e-691fcc073a34","Type":"ContainerStarted","Data":"816423188e6f1a243dc28491675b98bf792431b641d4c907b15eda674e325d99"}
Oct 29 23:13:13.136933 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.136904    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:13.140340 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.140315    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp" event={"ID":"37f1ad0a-20e9-45bb-9340-37f5dc7410d3","Type":"ContainerStarted","Data":"c9fa76a6969de6187140212e3528e5b65e8156ed7e8fcaf6d6f9c0a6664628c7"}
Oct 29 23:13:13.140437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.140360    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-q47dp" event={"ID":"37f1ad0a-20e9-45bb-9340-37f5dc7410d3","Type":"ContainerStarted","Data":"24fc8ba4bd9b3ff185dc8fc064a0723295c36d02ce52be26ecef2bae8b6717da"}
Oct 29 23:13:13.327046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.327009    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:13.665099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.665040    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:13.813276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813245    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:13.813400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813286    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:13.813467 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813397    5111 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:13.813467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813434    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:13.813573 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813480    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls podName:22482cc8-beda-4302-b88e-fed36f1bb7db nodeName:}" failed. No retries permitted until 2026-10-29 23:13:17.813461252 +0000 UTC m=+8.899744390 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mcxnv" (UID: "22482cc8-beda-4302-b88e-fed36f1bb7db") : secret "insights-runtime-extractor-tls" not found
Oct 29 23:13:13.813573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813520    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:13.813573 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813533    5111 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Oct 29 23:13:13.813573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813556    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:13.813774 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813617    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert podName:47425113-93db-453d-ac5e-b1b778b9bfe3 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:17.813597724 +0000 UTC m=+8.899880838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-n72kb" (UID: "47425113-93db-453d-ac5e-b1b778b9bfe3") : secret "networking-console-plugin-cert" not found
Oct 29 23:13:13.813774 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813628    5111 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Oct 29 23:13:13.813774 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813658    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:17.813648908 +0000 UTC m=+8.899932051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : secret "router-metrics-certs-default" not found
Oct 29 23:13:13.813774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813680    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:13.813774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.813740    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:13.814043 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:13.813801    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle podName:d7081746-11b4-4ea2-b641-2fc60782a2cf nodeName:}" failed. No retries permitted until 2026-10-29 23:13:17.813791047 +0000 UTC m=+8.900074192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle") pod "router-default-cddcfffcc-8wmcx" (UID: "d7081746-11b4-4ea2-b641-2fc60782a2cf") : configmap references non-existent config key: service-ca.crt
Oct 29 23:13:13.815811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.815782    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/aae915ed-0747-410a-acec-a18068a98efe-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-9wdnw\" (UID: \"aae915ed-0747-410a-acec-a18068a98efe\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:13.815921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.815835    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"image-registry-78868f4995-5rhvn\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") " pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:13.816091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.816070    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"image-registry-54576fc98-lv7kk\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") " pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:13.871732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.871710    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:13.959599 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.959538    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"
Oct 29 23:13:13.960707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.960683    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:13.970425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:13.970402    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:14.064692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.064549    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:14.144837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.144809    5111 generic.go:356] "Generic (PLEG): container finished" podID="e61d9ca2-dabd-4378-bee7-ecacbfd366a6" containerID="1b53185c777da105ce927a63497334004ee1c15db29729f98555da099a9e5feb" exitCode=0
Oct 29 23:13:14.145285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.144880    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-n4774" event={"ID":"e61d9ca2-dabd-4378-bee7-ecacbfd366a6","Type":"ContainerDied","Data":"1b53185c777da105ce927a63497334004ee1c15db29729f98555da099a9e5feb"}
Oct 29 23:13:14.145285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.145271    5111 scope.go:122] "RemoveContainer" containerID="1b53185c777da105ce927a63497334004ee1c15db29729f98555da099a9e5feb"
Oct 29 23:13:14.147211 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.147185    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-nggm8" event={"ID":"e525b897-14cf-4e83-a016-60ab539f490d","Type":"ContainerStarted","Data":"49d746ac5a4e8ce77314c085fde4a07dc3e52b10be8d4b7d4353689b3940e725"}
Oct 29 23:13:14.149715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.149622    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlgv" event={"ID":"303a5fa2-fea5-4e46-91d2-f8b8ae3ce57e","Type":"ContainerStarted","Data":"e96167314cdecfd9a9e7b397bc35ecfb0771995bab7a4bd69a24bc305e4346e5"}
Oct 29 23:13:14.257783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.257715    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:14.291122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.291103    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw"]
Oct 29 23:13:14.293911 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:14.293874    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae915ed_0747_410a_acec_a18068a98efe.slice/crio-726d476453fe97bec839d8d4c7dc490351de42d28a0022ba34a0091aaaccc085 WatchSource:0}: Error finding container 726d476453fe97bec839d8d4c7dc490351de42d28a0022ba34a0091aaaccc085: Status 404 returned error can't find the container with id 726d476453fe97bec839d8d4c7dc490351de42d28a0022ba34a0091aaaccc085
Oct 29 23:13:14.295793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.295771    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:13:14.298964 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:14.298902    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52c5a942_07fa_4c10_8dd0_85bf3e9226f5.slice/crio-cafbdad68c64045a8364f093761ece3e96b9480baaa66be01ccc911ac7792a29 WatchSource:0}: Error finding container cafbdad68c64045a8364f093761ece3e96b9480baaa66be01ccc911ac7792a29: Status 404 returned error can't find the container with id cafbdad68c64045a8364f093761ece3e96b9480baaa66be01ccc911ac7792a29
Oct 29 23:13:14.312438 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.312416    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:13:14.313462 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:14.313436    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6b5a784_3628_4301_a7c5_f193012659a0.slice/crio-6f9db91cb71eb165d00965b29d495239ffbc524b56f4c32ad41d0bc821aa4b99 WatchSource:0}: Error finding container 6f9db91cb71eb165d00965b29d495239ffbc524b56f4c32ad41d0bc821aa4b99: Status 404 returned error can't find the container with id 6f9db91cb71eb165d00965b29d495239ffbc524b56f4c32ad41d0bc821aa4b99
Oct 29 23:13:14.462353 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.462319    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:14.852618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.852173    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:14.999399 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:14.999367    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:15.156397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.156309    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78868f4995-5rhvn" event={"ID":"52c5a942-07fa-4c10-8dd0-85bf3e9226f5","Type":"ContainerStarted","Data":"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"}
Oct 29 23:13:15.156397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.156366    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78868f4995-5rhvn" event={"ID":"52c5a942-07fa-4c10-8dd0-85bf3e9226f5","Type":"ContainerStarted","Data":"cafbdad68c64045a8364f093761ece3e96b9480baaa66be01ccc911ac7792a29"}
Oct 29 23:13:15.156900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.156451    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:15.157877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.157850    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw" event={"ID":"aae915ed-0747-410a-acec-a18068a98efe","Type":"ContainerStarted","Data":"726d476453fe97bec839d8d4c7dc490351de42d28a0022ba34a0091aaaccc085"}
Oct 29 23:13:15.160036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.160012    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-n4774" event={"ID":"e61d9ca2-dabd-4378-bee7-ecacbfd366a6","Type":"ContainerStarted","Data":"149f644940f27e5dd37520b03c105b3eebe316a806645384859a4aefdc33319a"}
Oct 29 23:13:15.161841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.161817    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" event={"ID":"f6b5a784-3628-4301-a7c5-f193012659a0","Type":"ContainerStarted","Data":"3acf95031a44fa797587aba556405831c738fc86e4899d5a981b5a36a1403b2d"}
Oct 29 23:13:15.161960 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.161853    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" event={"ID":"f6b5a784-3628-4301-a7c5-f193012659a0","Type":"ContainerStarted","Data":"6f9db91cb71eb165d00965b29d495239ffbc524b56f4c32ad41d0bc821aa4b99"}
Oct 29 23:13:15.694341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.694306    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:15.859132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:15.859097    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.081993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.081948    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.167522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.167484    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-9wdnw" event={"ID":"aae915ed-0747-410a-acec-a18068a98efe","Type":"ContainerStarted","Data":"a2c7eddf32c7ded63c22ac093ad07cb7c8aea24918e32d72e0046c7d5794f554"}
Oct 29 23:13:16.167963 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.167834    5111 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:16.250195 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.250151    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.389831 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.389768    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.532727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.532693    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.694383 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.694320    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.840634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.840606    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:16.975359 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:16.975294    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:17.275570 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.275490    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:17.851908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.851875    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:17.852077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.851921    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:17.852077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.851943    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:17.852709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.852457    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:17.853117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.853081    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7081746-11b4-4ea2-b641-2fc60782a2cf-service-ca-bundle\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:17.855646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.855608    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/22482cc8-beda-4302-b88e-fed36f1bb7db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mcxnv\" (UID: \"22482cc8-beda-4302-b88e-fed36f1bb7db\") " pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:17.858540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.858496    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/47425113-93db-453d-ac5e-b1b778b9bfe3-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-n72kb\" (UID: \"47425113-93db-453d-ac5e-b1b778b9bfe3\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:17.859140 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.859117    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7081746-11b4-4ea2-b641-2fc60782a2cf-metrics-certs\") pod \"router-default-cddcfffcc-8wmcx\" (UID: \"d7081746-11b4-4ea2-b641-2fc60782a2cf\") " pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:17.862325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.862301    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb"
Oct 29 23:13:17.874144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:17.874110    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mcxnv"
Oct 29 23:13:18.156383 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.156357    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-n72kb"]
Oct 29 23:13:18.157161 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:18.157131    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47425113_93db_453d_ac5e_b1b778b9bfe3.slice/crio-03cdcdbb4e6056eddf752b5555e4343e851a8e8dc89db8a98943e5722793d8a0 WatchSource:0}: Error finding container 03cdcdbb4e6056eddf752b5555e4343e851a8e8dc89db8a98943e5722793d8a0: Status 404 returned error can't find the container with id 03cdcdbb4e6056eddf752b5555e4343e851a8e8dc89db8a98943e5722793d8a0
Oct 29 23:13:18.159133 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.159110    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:18.173072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.173048    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mcxnv"]
Oct 29 23:13:18.173745 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:18.173722    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22482cc8_beda_4302_b88e_fed36f1bb7db.slice/crio-940ebfca1dfe277678ea96fc0ddc3b6e4c4cd6b55ba31bcd6f536899f3400763 WatchSource:0}: Error finding container 940ebfca1dfe277678ea96fc0ddc3b6e4c4cd6b55ba31bcd6f536899f3400763: Status 404 returned error can't find the container with id 940ebfca1dfe277678ea96fc0ddc3b6e4c4cd6b55ba31bcd6f536899f3400763
Oct 29 23:13:18.177608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.177587    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb" event={"ID":"47425113-93db-453d-ac5e-b1b778b9bfe3","Type":"ContainerStarted","Data":"03cdcdbb4e6056eddf752b5555e4343e851a8e8dc89db8a98943e5722793d8a0"}
Oct 29 23:13:18.244573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.244549    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:18.454683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:18.454662    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cddcfffcc-8wmcx"]
Oct 29 23:13:18.456273 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:18.456251    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7081746_11b4_4ea2_b641_2fc60782a2cf.slice/crio-7a93028faeae59ab41381b52c122418913c9f83f45e012cc41ab4e87d7717e5c WatchSource:0}: Error finding container 7a93028faeae59ab41381b52c122418913c9f83f45e012cc41ab4e87d7717e5c: Status 404 returned error can't find the container with id 7a93028faeae59ab41381b52c122418913c9f83f45e012cc41ab4e87d7717e5c
Oct 29 23:13:19.183713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:19.183684    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mcxnv" event={"ID":"22482cc8-beda-4302-b88e-fed36f1bb7db","Type":"ContainerStarted","Data":"e4ff228fbb3eac99a141656927087b703ad52ffb492abec87ca7e14dcdea306d"}
Oct 29 23:13:19.183863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:19.183746    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mcxnv" event={"ID":"22482cc8-beda-4302-b88e-fed36f1bb7db","Type":"ContainerStarted","Data":"e83fe0357f6e387683bce32477c82b06d903c49ba62125c98e2b7adc57599fc6"}
Oct 29 23:13:19.183863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:19.183775    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mcxnv" event={"ID":"22482cc8-beda-4302-b88e-fed36f1bb7db","Type":"ContainerStarted","Data":"940ebfca1dfe277678ea96fc0ddc3b6e4c4cd6b55ba31bcd6f536899f3400763"}
Oct 29 23:13:19.186721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:19.186699    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cddcfffcc-8wmcx" event={"ID":"d7081746-11b4-4ea2-b641-2fc60782a2cf","Type":"ContainerStarted","Data":"a238c7c6c076f9d590dca14eaf0deca54a008bd23931bd37feaca7d3b5f3337b"}
Oct 29 23:13:19.186826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:19.186745    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cddcfffcc-8wmcx" event={"ID":"d7081746-11b4-4ea2-b641-2fc60782a2cf","Type":"ContainerStarted","Data":"7a93028faeae59ab41381b52c122418913c9f83f45e012cc41ab4e87d7717e5c"}
Oct 29 23:13:20.031893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:20.030870    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jdnh8"
Oct 29 23:13:20.159847 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:20.159825    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:20.162483 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:20.162461    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:20.191913 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:20.191880    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-n72kb" event={"ID":"47425113-93db-453d-ac5e-b1b778b9bfe3","Type":"ContainerStarted","Data":"11a991a4b322e253969f9fcfc2b88f0732f37e5dc6d1beff1946e844ee8829f4"}
Oct 29 23:13:20.193903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:20.193881    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mcxnv" event={"ID":"22482cc8-beda-4302-b88e-fed36f1bb7db","Type":"ContainerStarted","Data":"4531cd8fda665944d1c2c138388f289f0e5e2746ed646ccb10540c1c1606f5cb"}
Oct 29 23:13:21.197510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:21.197478    5111 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:23.754443 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:23.754412    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:24.168407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:24.168369    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f8xtp"
Oct 29 23:13:25.012414 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.012382    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:25.013111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.013090    5111 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 29 23:13:25.016430 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.016407    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-cddcfffcc-8wmcx"
Oct 29 23:13:25.222480 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.222446    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-pfvpc-drfj5"]
Oct 29 23:13:25.227502 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.227478    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.259146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.259081    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-pfvpc-drfj5"]
Oct 29 23:13:25.261142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.261115    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"]
Oct 29 23:13:25.267193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.267129    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:13:25.267301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.267273    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:25.282149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.280937    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"]
Oct 29 23:13:25.293268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.292937    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-74hgp\""
Oct 29 23:13:25.294286 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.294262    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 29 23:13:25.325889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.325858    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.326098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.326033    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.326098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.326083    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8db78\" (UniqueName: \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.326218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.326138    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.348141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.348114    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-zhng8"]
Oct 29 23:13:25.351815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.351792    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74d994776-2n5gg"]
Oct 29 23:13:25.351917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.351903    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.355485 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.355463    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.371025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.370957    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 29 23:13:25.371145 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.371009    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 29 23:13:25.371281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.371264    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-zhng8"]
Oct 29 23:13:25.374639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.374612    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:25.375290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.375269    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d994776-2n5gg"]
Oct 29 23:13:25.402597 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.402572    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-bwv5l\""
Oct 29 23:13:25.426516 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.426490    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.426642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.426536    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1eef2e78-c3b9-4f5f-bc38-33e67ef18f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-rpvlz\" (UID: \"1eef2e78-c3b9-4f5f-bc38-33e67ef18f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:25.426702 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.426678    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8db78\" (UniqueName: \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.426752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.426723    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.426807 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.426779    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.427539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.427460    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.427916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.427900    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.428030 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.428016    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.451857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.451834    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8db78\" (UniqueName: \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\") pod \"periodic-gathering-pfvpc-drfj5\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") " pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.539126 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.533334    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:25.539368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539342    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-bound-sa-token\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539410    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64e184bf-833d-4628-826e-001799a103b5-tmp\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.539486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539467    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-tls\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539560 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539490    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rswtt\" (UniqueName: \"kubernetes.io/projected/64e184bf-833d-4628-826e-001799a103b5-kube-api-access-rswtt\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.539560 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539515    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1eef2e78-c3b9-4f5f-bc38-33e67ef18f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-rpvlz\" (UID: \"1eef2e78-c3b9-4f5f-bc38-33e67ef18f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:25.539560 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539531    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-installation-pull-secrets\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539560 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539548    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/060116d8-5fc1-43b6-9f4d-0ffc236c4372-ca-trust-extracted\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539730 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539659    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-image-registry-private-configuration\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539758    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsn2d\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-kube-api-access-fsn2d\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539797    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-trusted-ca\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.539925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.539896    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-certificates\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.541858 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.541841    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1eef2e78-c3b9-4f5f-bc38-33e67ef18f35-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-rpvlz\" (UID: \"1eef2e78-c3b9-4f5f-bc38-33e67ef18f35\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:25.543729 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.543713    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:13:25.593009 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.592108    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:25.640692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640658    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rswtt\" (UniqueName: \"kubernetes.io/projected/64e184bf-833d-4628-826e-001799a103b5-kube-api-access-rswtt\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.640692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640696    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-installation-pull-secrets\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.640692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640714    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/060116d8-5fc1-43b6-9f4d-0ffc236c4372-ca-trust-extracted\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640732    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-image-registry-private-configuration\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640778    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsn2d\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-kube-api-access-fsn2d\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640802    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-trusted-ca\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640818    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-certificates\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640840    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-bound-sa-token\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640883    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64e184bf-833d-4628-826e-001799a103b5-tmp\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.641020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.640906    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-tls\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.643901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.642887    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-trusted-ca\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.643901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.643587    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64e184bf-833d-4628-826e-001799a103b5-tmp\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.643901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.643628    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/060116d8-5fc1-43b6-9f4d-0ffc236c4372-ca-trust-extracted\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.643901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.643640    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-certificates\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.645612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.645590    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-image-registry-private-configuration\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.647028 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.646986    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-registry-tls\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.648467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.648427    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/060116d8-5fc1-43b6-9f4d-0ffc236c4372-installation-pull-secrets\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.666920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.666876    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-bound-sa-token\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.671614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.671237    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsn2d\" (UniqueName: \"kubernetes.io/projected/060116d8-5fc1-43b6-9f4d-0ffc236c4372-kube-api-access-fsn2d\") pod \"image-registry-74d994776-2n5gg\" (UID: \"060116d8-5fc1-43b6-9f4d-0ffc236c4372\") " pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.671614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.671384    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:25.683749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.683732    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rswtt\" (UniqueName: \"kubernetes.io/projected/64e184bf-833d-4628-826e-001799a103b5-kube-api-access-rswtt\") pod \"downloads-6dd789c8b7-zhng8\" (UID: \"64e184bf-833d-4628-826e-001799a103b5\") " pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.708424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.708364    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:25.851191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.851164    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-pfvpc-drfj5"]
Oct 29 23:13:25.852738 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:25.852715    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b2cc3d_979f_465b_b5c2_d4b313aeeddb.slice/crio-14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3 WatchSource:0}: Error finding container 14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3: Status 404 returned error can't find the container with id 14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3
Oct 29 23:13:25.914182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.914159    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"]
Oct 29 23:13:25.916623 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:25.916589    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eef2e78_c3b9_4f5f_bc38_33e67ef18f35.slice/crio-9a62f53125736e4585cbf7d797d6220ef088b0ff98f697216fd0361c3c2f110a WatchSource:0}: Error finding container 9a62f53125736e4585cbf7d797d6220ef088b0ff98f697216fd0361c3c2f110a: Status 404 returned error can't find the container with id 9a62f53125736e4585cbf7d797d6220ef088b0ff98f697216fd0361c3c2f110a
Oct 29 23:13:25.964315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.964279    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:25.997618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:25.997587    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74d994776-2n5gg"]
Oct 29 23:13:26.146932 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.146219    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b9h9g"
Oct 29 23:13:26.164398 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:26.164370    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:13:26.227002 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.226945    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d994776-2n5gg" event={"ID":"060116d8-5fc1-43b6-9f4d-0ffc236c4372","Type":"ContainerStarted","Data":"6036c5c7241920c2f923ccb6b6f8a5fb8b4a239c8158a79ced368eddcb4e2a9c"}
Oct 29 23:13:26.227142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.227033    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74d994776-2n5gg" event={"ID":"060116d8-5fc1-43b6-9f4d-0ffc236c4372","Type":"ContainerStarted","Data":"b4155b939d5acbbbc0264200cea9643be05999c4591ec567d33c65f3c04df5c9"}
Oct 29 23:13:26.227142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.227068    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:26.228617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.228593    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz" event={"ID":"1eef2e78-c3b9-4f5f-bc38-33e67ef18f35","Type":"ContainerStarted","Data":"9a62f53125736e4585cbf7d797d6220ef088b0ff98f697216fd0361c3c2f110a"}
Oct 29 23:13:26.230284 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.230256    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pfvpc-drfj5" event={"ID":"95b2cc3d-979f-465b-b5c2-d4b313aeeddb","Type":"ContainerStarted","Data":"5f8a0a28dbb5dad9d82697fdcd77423cd615b294a682de52570253bfabe57fcd"}
Oct 29 23:13:26.230374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.230310    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pfvpc-drfj5" event={"ID":"95b2cc3d-979f-465b-b5c2-d4b313aeeddb","Type":"ContainerStarted","Data":"14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3"}
Oct 29 23:13:26.271642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.271587    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74d994776-2n5gg" podStartSLOduration=1.271571006 podStartE2EDuration="1.271571006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:26.269922378 +0000 UTC m=+17.356205512" watchObservedRunningTime="2026-10-29 23:13:26.271571006 +0000 UTC m=+17.357854118"
Oct 29 23:13:26.284869 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.284842    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-zhng8"]
Oct 29 23:13:26.289201 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:26.289174    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e184bf_833d_4628_826e_001799a103b5.slice/crio-cf42be569a7672868e3b12b7fe4378fa37465319f1e9ca334be62549ed56ed04 WatchSource:0}: Error finding container cf42be569a7672868e3b12b7fe4378fa37465319f1e9ca334be62549ed56ed04: Status 404 returned error can't find the container with id cf42be569a7672868e3b12b7fe4378fa37465319f1e9ca334be62549ed56ed04
Oct 29 23:13:26.309075 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.309013    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-pfvpc-drfj5" podStartSLOduration=1.308999027 podStartE2EDuration="1.308999027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:26.307963864 +0000 UTC m=+17.394246997" watchObservedRunningTime="2026-10-29 23:13:26.308999027 +0000 UTC m=+17.395282157"
Oct 29 23:13:26.828679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.828654    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:26.860004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.859962    5111 patch_prober.go:29] interesting pod/image-registry-54576fc98-lv7kk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 29 23:13:26.860119 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.860048    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:13:26.970639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:26.970604    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:27.237055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:27.236949    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-zhng8" event={"ID":"64e184bf-833d-4628-826e-001799a103b5","Type":"ContainerStarted","Data":"cf42be569a7672868e3b12b7fe4378fa37465319f1e9ca334be62549ed56ed04"}
Oct 29 23:13:27.238737 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:27.238702    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz" event={"ID":"1eef2e78-c3b9-4f5f-bc38-33e67ef18f35","Type":"ContainerStarted","Data":"8110424d488192d0de566fcb4d9950039763e2a07d8ebad0be273af0bf89786f"}
Oct 29 23:13:27.277053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:27.276990    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz" podStartSLOduration=1.368027217 podStartE2EDuration="2.276959803s" totalImagesPullingTime="908.932586ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:27.276176682 +0000 UTC m=+18.362459816" watchObservedRunningTime="2026-10-29 23:13:27.276959803 +0000 UTC m=+18.363242938"
Oct 29 23:13:28.244486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:28.244450    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:28.249816 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:28.249789    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-rpvlz"
Oct 29 23:13:28.764453 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:28.764411    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:28.911555 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:28.911516    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:29.062536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:29.062468    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:29.862862 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:29.862823    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:30.019717 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:30.019682    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:30.559584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:30.559553    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-n72kb_47425113-93db-453d-ac5e-b1b778b9bfe3/networking-console-plugin/0.log"
Oct 29 23:13:30.763715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:30.763682    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:13:30.962240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:30.962209    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/2.log"
Oct 29 23:13:31.351774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:31.351742    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:31.759878 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:31.759793    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8xtp_d42a6856-295d-4cf1-8a6e-691fcc073a34/dns/0.log"
Oct 29 23:13:31.788074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:31.788047    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:31.961275 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:31.961243    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8xtp_d42a6856-295d-4cf1-8a6e-691fcc073a34/kube-rbac-proxy/0.log"
Oct 29 23:13:32.560550 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:32.560521    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4blvf_e5f497d2-8302-4707-b479-4dca872829e3/dns-node-resolver/0.log"
Oct 29 23:13:33.042857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.042822    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.164874 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.164842    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-54576fc98-lv7kk_f6b5a784-3628-4301-a7c5-f193012659a0/registry/0.log"
Oct 29 23:13:33.204063 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.204034    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.361771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.361740    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74d994776-2n5gg_060116d8-5fc1-43b6-9f4d-0ffc236c4372/registry/0.log"
Oct 29 23:13:33.365809 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.365781    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.488085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.488051    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b8zsd"]
Oct 29 23:13:33.525378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.525352    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.540148 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.540119    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.553140 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.553115    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vwjx7\""
Oct 29 23:13:33.553269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.553160    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 29 23:13:33.553269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.553174    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 29 23:13:33.553269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.553186    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 29 23:13:33.553269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.553253    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 29 23:13:33.566527 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.566488    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-78868f4995-5rhvn_52c5a942-07fa-4c10-8dd0-85bf3e9226f5/registry/0.log"
Oct 29 23:13:33.619126 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619065    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-wtmp\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619126 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619105    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619132    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9jrf\" (UniqueName: \"kubernetes.io/projected/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-kube-api-access-w9jrf\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619289    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-textfile\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619332    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-root\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619358    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619447    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-tls\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619504    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-metrics-client-ca\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.619581 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.619539    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-sys\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.681539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.681515    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.719951 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.719923    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-textfile\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720014    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-root\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720041    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720071    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-tls\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720092    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-metrics-client-ca\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720126    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-sys\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720182    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-wtmp\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720197    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720216    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9jrf\" (UniqueName: \"kubernetes.io/projected/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-kube-api-access-w9jrf\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720636 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720619    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-wtmp\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720758 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720738    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-root\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.720852 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.720837    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-metrics-client-ca\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.721074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.721054    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-textfile\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.721218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.721123    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-sys\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.721218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.721165    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-accelerators-collector-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.722915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.722887    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-tls\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.723233 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.723205    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.740561 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.740527    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9jrf\" (UniqueName: \"kubernetes.io/projected/9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d-kube-api-access-w9jrf\") pod \"node-exporter-b8zsd\" (UID: \"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d\") " pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.759876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.759853    5111 patch_prober.go:29] interesting pod/image-registry-54576fc98-lv7kk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Oct 29 23:13:33.759984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.759908    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:13:33.760146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.760130    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9m5t_d8081555-b5de-4ffa-bbe7-e2931b269310/node-ca/0.log"
Oct 29 23:13:33.841200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.841174    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:33.853202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:33.853179    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b8zsd"
Oct 29 23:13:33.886182 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:13:33.886155    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d84fbaf_f8de_4a9d_9b1d_f0fe11601c6d.slice/crio-e1f0d92c1a74f4f7211e0130103601b51492364c633c05b1be35fab3d418de00 WatchSource:0}: Error finding container e1f0d92c1a74f4f7211e0130103601b51492364c633c05b1be35fab3d418de00: Status 404 returned error can't find the container with id e1f0d92c1a74f4f7211e0130103601b51492364c633c05b1be35fab3d418de00
Oct 29 23:13:34.048051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.048015    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:34.197608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.197513    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:34.273899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.273865    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b8zsd" event={"ID":"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d","Type":"ContainerStarted","Data":"e1f0d92c1a74f4f7211e0130103601b51492364c633c05b1be35fab3d418de00"}
Oct 29 23:13:34.362904 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.362877    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cddcfffcc-8wmcx_d7081746-11b4-4ea2-b641-2fc60782a2cf/router/0.log"
Oct 29 23:13:34.647679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.647636    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:34.761344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.761312    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hvb2t_f63a0452-9661-4fbb-9396-9ac7b5524dd3/serve-healthcheck-canary/0.log"
Oct 29 23:13:34.930527 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:34.930410    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:35.157127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:35.157088    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:35.274656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:35.274633    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:35.962254 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:35.962219    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:36.168648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:36.168612    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:36.285085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:36.285004    5111 generic.go:356] "Generic (PLEG): container finished" podID="9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d" containerID="6c4c63f85411cad4ac905d89ecfab7f5cd519523c650adeb1df0e82c2318c818" exitCode=0
Oct 29 23:13:36.285239 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:36.285100    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b8zsd" event={"ID":"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d","Type":"ContainerDied","Data":"6c4c63f85411cad4ac905d89ecfab7f5cd519523c650adeb1df0e82c2318c818"}
Oct 29 23:13:36.741766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:36.741717    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:36.883505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:36.883472    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:37.292617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:37.292584    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b8zsd" event={"ID":"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d","Type":"ContainerStarted","Data":"1f4a3995057ea11ee47474997bb1e10766e7992bc50975226e2355153ac152fb"}
Oct 29 23:13:37.293089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:37.292640    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b8zsd" event={"ID":"9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d","Type":"ContainerStarted","Data":"daf6034a692cdab897b5c65e9d0b376a64c8f045c633159839682bdbf9aedb69"}
Oct 29 23:13:37.357554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:37.357506    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b8zsd" podStartSLOduration=2.926344218 podStartE2EDuration="4.357494495s" totalImagesPullingTime="1.344067038s" totalInitContainerRuntime="87.083239ms" isStatefulPod=true podCreationTimestamp="2026-10-29 23:13:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:37.345212113 +0000 UTC m=+28.431495248" watchObservedRunningTime="2026-10-29 23:13:37.357494495 +0000 UTC m=+28.443777628"
Oct 29 23:13:37.645912 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:37.645877    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:37.784936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:37.784891    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:38.246999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:38.246949    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-q9xkk"
Oct 29 23:13:38.285936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:38.285906    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:38.432096 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:38.432058    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:40.474709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:40.474672    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:40.946633 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:40.946601    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:42.321899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.321789    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-zhng8" event={"ID":"64e184bf-833d-4628-826e-001799a103b5","Type":"ContainerStarted","Data":"936b28ad660d4c18575b5ef5117ad43482ec781de3c191ea72e9c353e7a6032e"}
Oct 29 23:13:42.322368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.322185    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:42.338937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.338916    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-zhng8"
Oct 29 23:13:42.387806 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.387749    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-zhng8" podStartSLOduration=1.742430522 podStartE2EDuration="17.387735688s" totalImagesPullingTime="15.645305166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:42.374543827 +0000 UTC m=+33.460826961" watchObservedRunningTime="2026-10-29 23:13:42.387735688 +0000 UTC m=+33.474018847"
Oct 29 23:13:42.606441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.606396    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:42.764599 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:42.764558    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:43.760066 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:43.760029    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:13:44.252140 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:44.252107    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:44.412989 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:44.412951    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:44.914567 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:44.914536    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:45.084022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:45.083987    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:46.329347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:46.329311    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:46.477763 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:46.477727    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:47.244262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:47.244231    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74d994776-2n5gg"
Oct 29 23:13:47.335931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:47.335899    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:13:47.541124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:47.541046    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:47.701526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:47.701479    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:48.214134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:48.214099    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:48.365440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:48.365409    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:49.873999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:49.873891    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:50.045211 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.045152    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:50.306641 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.306558    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78868f4995-5rhvn" podUID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" containerName="registry" containerID="cri-o://48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7" gracePeriod=30
Oct 29 23:13:50.625149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.625123    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:50.687781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.687749    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.687961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.687824    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.687961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.687871    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.687961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.687942    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.688154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688010    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.688154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688055    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.688154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688086    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.688154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688112    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") pod \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\" (UID: \"52c5a942-07fa-4c10-8dd0-85bf3e9226f5\") "
Oct 29 23:13:50.688513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688369    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:50.688862 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.688827    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:50.690509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.690480    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:50.690630 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.690516    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:50.691037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.691012    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:13:50.691140 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.691120    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:50.691392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.691369    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "kube-api-access-clfb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:50.697577 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.697542    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted" pod "52c5a942-07fa-4c10-8dd0-85bf3e9226f5" (UID: "52c5a942-07fa-4c10-8dd0-85bf3e9226f5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:13:50.788916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788885    5111 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-trusted-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.788916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788912    5111 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-installation-pull-secrets\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.788916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788923    5111 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-ca-trust-extracted\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.789166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788933    5111 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-certificates\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.789166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788942    5111 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-bound-sa-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.789166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788951    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clfb4\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-kube-api-access-clfb4\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.789166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.788960    5111 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-registry-tls\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:50.789558 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:50.789541    5111 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/52c5a942-07fa-4c10-8dd0-85bf3e9226f5-image-registry-private-configuration\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:51.384426 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.384393    5111 generic.go:356] "Generic (PLEG): container finished" podID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" containerID="48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7" exitCode=0
Oct 29 23:13:51.384924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.384450    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78868f4995-5rhvn"
Oct 29 23:13:51.384924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.384473    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78868f4995-5rhvn" event={"ID":"52c5a942-07fa-4c10-8dd0-85bf3e9226f5","Type":"ContainerDied","Data":"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"}
Oct 29 23:13:51.384924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.384520    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78868f4995-5rhvn" event={"ID":"52c5a942-07fa-4c10-8dd0-85bf3e9226f5","Type":"ContainerDied","Data":"cafbdad68c64045a8364f093761ece3e96b9480baaa66be01ccc911ac7792a29"}
Oct 29 23:13:51.384924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.384554    5111 scope.go:122] "RemoveContainer" containerID="48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"
Oct 29 23:13:51.398518 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.398493    5111 scope.go:122] "RemoveContainer" containerID="48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"
Oct 29 23:13:51.401311 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:51.401278    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7\": container with ID starting with 48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7 not found: ID does not exist" containerID="48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"
Oct 29 23:13:51.401418 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.401323    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7"} err="failed to get container status \"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7\": rpc error: code = NotFound desc = could not find container \"48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7\": container with ID starting with 48e32e838b72c7d4f99ba022ea0c842fcfad3134164d962aee058bd822482ca7 not found: ID does not exist"
Oct 29 23:13:51.455180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.455145    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:51.455387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.455362    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:13:51.466764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.466740    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78868f4995-5rhvn"]
Oct 29 23:13:51.637164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:51.637086    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:52.249746 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:52.249711    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:52.409819 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:52.409789    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:53.060550 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:53.060500    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" path="/var/lib/kubelet/pods/52c5a942-07fa-4c10-8dd0-85bf3e9226f5/volumes"
Oct 29 23:13:55.218888 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:55.218846    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:55.381583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:55.381552    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:56.149144 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:56.149105    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:13:57.571750 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:57.571715    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:57.650749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:57.650721    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:57.650952 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:57.650930    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-fth5d-rckp7" podUID="de6957c0-aaa9-449b-b5be-4f29212f40ce" containerName="insights-gathering" containerID="cri-o://f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb" gracePeriod=30
Oct 29 23:13:57.731387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:57.731362    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:57.944381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:57.944353    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:58.054476 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054439    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") pod \"de6957c0-aaa9-449b-b5be-4f29212f40ce\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") "
Oct 29 23:13:58.054639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054495    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") pod \"de6957c0-aaa9-449b-b5be-4f29212f40ce\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") "
Oct 29 23:13:58.054639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054560    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") pod \"de6957c0-aaa9-449b-b5be-4f29212f40ce\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") "
Oct 29 23:13:58.054639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054612    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") pod \"de6957c0-aaa9-449b-b5be-4f29212f40ce\" (UID: \"de6957c0-aaa9-449b-b5be-4f29212f40ce\") "
Oct 29 23:13:58.054940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054910    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle" pod "de6957c0-aaa9-449b-b5be-4f29212f40ce" (UID: "de6957c0-aaa9-449b-b5be-4f29212f40ce"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:58.055048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054957    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle" pod "de6957c0-aaa9-449b-b5be-4f29212f40ce" (UID: "de6957c0-aaa9-449b-b5be-4f29212f40ce"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:13:58.055048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.054958    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path" pod "de6957c0-aaa9-449b-b5be-4f29212f40ce" (UID: "de6957c0-aaa9-449b-b5be-4f29212f40ce"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:13:58.056763 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.056742    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g" pod "de6957c0-aaa9-449b-b5be-4f29212f40ce" (UID: "de6957c0-aaa9-449b-b5be-4f29212f40ce"). InnerVolumeSpecName "kube-api-access-c254g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:13:58.106788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.106723    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:58.155657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.155635    5111 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/de6957c0-aaa9-449b-b5be-4f29212f40ce-archives-path\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.155657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.155656    5111 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-trusted-ca-bundle\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.155800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.155670    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c254g\" (UniqueName: \"kubernetes.io/projected/de6957c0-aaa9-449b-b5be-4f29212f40ce-kube-api-access-c254g\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.155800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.155680    5111 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/de6957c0-aaa9-449b-b5be-4f29212f40ce-service-ca-bundle\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:13:58.288786 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.288762    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:13:58.422486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.422421    5111 generic.go:356] "Generic (PLEG): container finished" podID="de6957c0-aaa9-449b-b5be-4f29212f40ce" containerID="f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb" exitCode=2
Oct 29 23:13:58.422486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.422471    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-fth5d-rckp7"
Oct 29 23:13:58.422640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.422509    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fth5d-rckp7" event={"ID":"de6957c0-aaa9-449b-b5be-4f29212f40ce","Type":"ContainerDied","Data":"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"}
Oct 29 23:13:58.422640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.422559    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-fth5d-rckp7" event={"ID":"de6957c0-aaa9-449b-b5be-4f29212f40ce","Type":"ContainerDied","Data":"39b2f08448bf3562e42767114a7e2d3c0445f073d4af9f1b2a8222ea17df1280"}
Oct 29 23:13:58.422640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.422586    5111 scope.go:122] "RemoveContainer" containerID="f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"
Oct 29 23:13:58.431559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.431541    5111 scope.go:122] "RemoveContainer" containerID="f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"
Oct 29 23:13:58.431823 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:13:58.431804    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb\": container with ID starting with f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb not found: ID does not exist" containerID="f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"
Oct 29 23:13:58.431886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.431836    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb"} err="failed to get container status \"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb\": rpc error: code = NotFound desc = could not find container \"f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb\": container with ID starting with f6265b2b7a4ef938907c66fe857b4c996224e02fb8f1e9d5fc99b281d58039bb not found: ID does not exist"
Oct 29 23:13:58.477637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.477613    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:58.489523 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:58.489499    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-fth5d-rckp7"]
Oct 29 23:13:59.060255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:59.060224    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de6957c0-aaa9-449b-b5be-4f29212f40ce" path="/var/lib/kubelet/pods/de6957c0-aaa9-449b-b5be-4f29212f40ce/volumes"
Oct 29 23:13:59.886759 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:13:59.886727    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:00.067667 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:00.067628    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:05.314147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:05.314115    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:05.456045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:05.456017    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:07.551085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:07.551054    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:07.693222 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:07.693190    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:07.865966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:07.865918    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:08.022070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:08.022042    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:11.090532 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:11.090496    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:11.255879 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:11.255853    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:12.360072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.360002    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry" containerID="cri-o://3acf95031a44fa797587aba556405831c738fc86e4899d5a981b5a36a1403b2d" gracePeriod=30
Oct 29 23:14:12.489642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.489616    5111 generic.go:356] "Generic (PLEG): container finished" podID="f6b5a784-3628-4301-a7c5-f193012659a0" containerID="3acf95031a44fa797587aba556405831c738fc86e4899d5a981b5a36a1403b2d" exitCode=0
Oct 29 23:14:12.489741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.489686    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" event={"ID":"f6b5a784-3628-4301-a7c5-f193012659a0","Type":"ContainerDied","Data":"3acf95031a44fa797587aba556405831c738fc86e4899d5a981b5a36a1403b2d"}
Oct 29 23:14:12.631628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.631573    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:14:12.662326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662303    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662377    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662407    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662453    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662525 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662489    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662525 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662517    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662555    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662603    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") pod \"f6b5a784-3628-4301-a7c5-f193012659a0\" (UID: \"f6b5a784-3628-4301-a7c5-f193012659a0\") "
Oct 29 23:14:12.662718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662622    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:12.662834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662812    5111 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-trusted-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.663035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.662956    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:12.664907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.664876    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "kube-api-access-5llz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:12.665130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.665107    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:12.665229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.665145    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:12.665363 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.665344    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:12.665414 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.665357    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:14:12.671409 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.671376    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted" pod "f6b5a784-3628-4301-a7c5-f193012659a0" (UID: "f6b5a784-3628-4301-a7c5-f193012659a0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:14:12.763875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763852    5111 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-installation-pull-secrets\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.763875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763876    5111 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-registry-tls\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.764031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763891    5111 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f6b5a784-3628-4301-a7c5-f193012659a0-ca-trust-extracted\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.764031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763901    5111 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f6b5a784-3628-4301-a7c5-f193012659a0-registry-certificates\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.764031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763910    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5llz7\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-kube-api-access-5llz7\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.764031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763919    5111 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f6b5a784-3628-4301-a7c5-f193012659a0-bound-sa-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:12.764031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:12.763929    5111 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f6b5a784-3628-4301-a7c5-f193012659a0-image-registry-private-configuration\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:13.498301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:13.498270    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54576fc98-lv7kk"
Oct 29 23:14:13.498301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:13.498292    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54576fc98-lv7kk" event={"ID":"f6b5a784-3628-4301-a7c5-f193012659a0","Type":"ContainerDied","Data":"6f9db91cb71eb165d00965b29d495239ffbc524b56f4c32ad41d0bc821aa4b99"}
Oct 29 23:14:13.498845 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:13.498354    5111 scope.go:122] "RemoveContainer" containerID="3acf95031a44fa797587aba556405831c738fc86e4899d5a981b5a36a1403b2d"
Oct 29 23:14:13.561751 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:13.561728    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:14:13.573945 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:13.573921    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54576fc98-lv7kk"]
Oct 29 23:14:14.645870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:14.645835    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:14.781069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:14.781033    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:15.060615 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:15.060548    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" path="/var/lib/kubelet/pods/f6b5a784-3628-4301-a7c5-f193012659a0/volumes"
Oct 29 23:14:26.151163 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:14:26.151122    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:14:29.852606 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:29.852561    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:29.985582 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:29.985542    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:38.604333 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:38.604294    5111 generic.go:356] "Generic (PLEG): container finished" podID="fe93ab9c-ae4e-48af-a423-74e32a601101" containerID="d2edeb105d387f401aa36ffab4f29d9243d2f142340925bf6ad6e194ff535f8f" exitCode=0
Oct 29 23:14:38.604776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:38.604381    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst" event={"ID":"fe93ab9c-ae4e-48af-a423-74e32a601101","Type":"ContainerDied","Data":"d2edeb105d387f401aa36ffab4f29d9243d2f142340925bf6ad6e194ff535f8f"}
Oct 29 23:14:38.604776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:38.604743    5111 scope.go:122] "RemoveContainer" containerID="d2edeb105d387f401aa36ffab4f29d9243d2f142340925bf6ad6e194ff535f8f"
Oct 29 23:14:39.610217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:39.610181    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-bpnst" event={"ID":"fe93ab9c-ae4e-48af-a423-74e32a601101","Type":"ContainerStarted","Data":"d0b9c88002c354e6cce16d98815c9b8b35acee3c7582c9ff5c5fc0d972bef855"}
Oct 29 23:14:54.676511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:54.676482    5111 generic.go:356] "Generic (PLEG): container finished" podID="95b2cc3d-979f-465b-b5c2-d4b313aeeddb" containerID="5f8a0a28dbb5dad9d82697fdcd77423cd615b294a682de52570253bfabe57fcd" exitCode=0
Oct 29 23:14:54.676877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:54.676560    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pfvpc-drfj5" event={"ID":"95b2cc3d-979f-465b-b5c2-d4b313aeeddb","Type":"ContainerDied","Data":"5f8a0a28dbb5dad9d82697fdcd77423cd615b294a682de52570253bfabe57fcd"}
Oct 29 23:14:55.803164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.803139    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:14:55.879612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.879588    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\") pod \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") "
Oct 29 23:14:55.879766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.879656    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\") pod \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") "
Oct 29 23:14:55.879766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.879682    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\" (UniqueName: \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\") pod \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") "
Oct 29 23:14:55.879766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.879742    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\") pod \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\" (UID: \"95b2cc3d-979f-465b-b5c2-d4b313aeeddb\") "
Oct 29 23:14:55.879962 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.879935    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path" pod "95b2cc3d-979f-465b-b5c2-d4b313aeeddb" (UID: "95b2cc3d-979f-465b-b5c2-d4b313aeeddb"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:14:55.880055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.880013    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle" pod "95b2cc3d-979f-465b-b5c2-d4b313aeeddb" (UID: "95b2cc3d-979f-465b-b5c2-d4b313aeeddb"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:55.880221 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.880201    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle" pod "95b2cc3d-979f-465b-b5c2-d4b313aeeddb" (UID: "95b2cc3d-979f-465b-b5c2-d4b313aeeddb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:14:55.881789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.881771    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78" pod "95b2cc3d-979f-465b-b5c2-d4b313aeeddb" (UID: "95b2cc3d-979f-465b-b5c2-d4b313aeeddb"). InnerVolumeSpecName "kube-api-access-8db78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:14:55.980991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.980928    5111 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-service-ca-bundle\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:55.980991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.980948    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8db78\" (UniqueName: \"kubernetes.io/projected/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-kube-api-access-8db78\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:55.980991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.980959    5111 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-trusted-ca-bundle\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:55.980991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:55.980968    5111 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/95b2cc3d-979f-465b-b5c2-d4b313aeeddb-archives-path\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:14:56.147608 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:14:56.147580    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:14:56.686020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:56.685963    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-pfvpc-drfj5"
Oct 29 23:14:56.686020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:56.686003    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-pfvpc-drfj5" event={"ID":"95b2cc3d-979f-465b-b5c2-d4b313aeeddb","Type":"ContainerDied","Data":"14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3"}
Oct 29 23:14:56.686260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:56.686057    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="14b51e91bcf6ea3f66625cf2e347dd36ac875dc3f95244fdd90eb009501ab0e3"
Oct 29 23:14:57.148676 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:57.148643    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:57.291514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:57.291461    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:14:58.246287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:14:58.246256    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:15:15.110499 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:15:15.110448    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:15:15.259065 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:15:15.259030    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:15:26.149038 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:15:26.148937    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:15:56.146043 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:15:56.145999    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:15:58.312028 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:15:58.311995    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:00.061710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:00.061666    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:20.414358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:20.414320    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:21.306119 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:21.306087    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:23.614391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:23.614357    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:23.785071 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:23.785034    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:26.149845 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:16:26.149800    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:16:26.167963 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:26.167940    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:26.333563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:26.333534    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:26.743229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:26.743196    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:26.958787 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:26.958757    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:28.758568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:28.758530    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:28.988510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:28.988476    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:30.613683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:30.613643    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:30.751289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:30.751254    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:34.844960 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:34.844926    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:35.010296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:35.010259    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:35.803610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:35.803572    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:38.895963 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:38.895910    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:39.030291 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:39.030254    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:45.265422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:45.265379    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:45.402919 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:45.402881    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:46.859759 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:46.859726    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:56.148648 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:16:56.148555    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:16:57.470522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:57.470487    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:57.619774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:57.619743    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:58.255099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:58.255064    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:58.397111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:58.397077    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:58.541511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:58.541445    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:59.017923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.017891    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:59.189199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189172    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"]
Oct 29 23:16:59.189460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189448    5111 cpu_manager.go:414] "containerMap: removing container" podUID="de6957c0-aaa9-449b-b5be-4f29212f40ce" containerName="insights-gathering"
Oct 29 23:16:59.189509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189461    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de6957c0-aaa9-449b-b5be-4f29212f40ce" containerName="insights-gathering"
Oct 29 23:16:59.189509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189476    5111 cpu_manager.go:414] "containerMap: removing container" podUID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" containerName="registry"
Oct 29 23:16:59.189509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189481    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" containerName="registry"
Oct 29 23:16:59.189509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189496    5111 cpu_manager.go:414] "containerMap: removing container" podUID="95b2cc3d-979f-465b-b5c2-d4b313aeeddb" containerName="insights-gathering"
Oct 29 23:16:59.189509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189502    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95b2cc3d-979f-465b-b5c2-d4b313aeeddb" containerName="insights-gathering"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189531    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189537    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189586    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="de6957c0-aaa9-449b-b5be-4f29212f40ce" containerName="insights-gathering"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189596    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="52c5a942-07fa-4c10-8dd0-85bf3e9226f5" containerName="registry"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189603    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="95b2cc3d-979f-465b-b5c2-d4b313aeeddb" containerName="insights-gathering"
Oct 29 23:16:59.189697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.189611    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6b5a784-3628-4301-a7c5-f193012659a0" containerName="registry"
Oct 29 23:16:59.191715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.191699    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.225160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.225128    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"]
Oct 29 23:16:59.289530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.289473    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 29 23:16:59.289530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.289476    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 29 23:16:59.289843 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.289824    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n9h54\""
Oct 29 23:16:59.289881 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.289838    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 29 23:16:59.290030 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.290012    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 29 23:16:59.290106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.290094    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 29 23:16:59.344810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.344780    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-metrics-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.344931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.344823    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-manifests\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.344931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.344850    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-tmp\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.345077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.345006    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.345077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.345045    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h5wm\" (UniqueName: \"kubernetes.io/projected/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-kube-api-access-8h5wm\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.345168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.345109    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.393866 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.393841    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:59.446349 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446327    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446355    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h5wm\" (UniqueName: \"kubernetes.io/projected/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-kube-api-access-8h5wm\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446378    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446396    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-metrics-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446527    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-manifests\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446590    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-tmp\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.446926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.446903    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-manifests\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.448784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.448759    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-tmp\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.449051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.449025    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-metrics-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.449370 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.449350    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-webhook-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.449370 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.449359    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.465808 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.465783    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h5wm\" (UniqueName: \"kubernetes.io/projected/d47b2c1f-2018-4906-bd8c-b44ee9b8240b-kube-api-access-8h5wm\") pod \"opendatahub-operator-controller-manager-bb7fb9d97-zd2fw\" (UID: \"d47b2c1f-2018-4906-bd8c-b44ee9b8240b\") " pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.505253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.505232    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:16:59.545718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.545658    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:59.693010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.692958    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:16:59.840151 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:16:59.840129    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"]
Oct 29 23:16:59.840307 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:16:59.840288    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd47b2c1f_2018_4906_bd8c_b44ee9b8240b.slice/crio-d89d7e4d64a40aef1d5abc3f815f709ce24a1590b89aed84cab53b61f9886157 WatchSource:0}: Error finding container d89d7e4d64a40aef1d5abc3f815f709ce24a1590b89aed84cab53b61f9886157: Status 404 returned error can't find the container with id d89d7e4d64a40aef1d5abc3f815f709ce24a1590b89aed84cab53b61f9886157
Oct 29 23:17:00.203153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:00.203076    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw" event={"ID":"d47b2c1f-2018-4906-bd8c-b44ee9b8240b","Type":"ContainerStarted","Data":"d89d7e4d64a40aef1d5abc3f815f709ce24a1590b89aed84cab53b61f9886157"}
Oct 29 23:17:00.797020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:00.796970    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:03.217723 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:03.217693    5111 generic.go:356] "Generic (PLEG): container finished" podID="d47b2c1f-2018-4906-bd8c-b44ee9b8240b" containerID="858d413e40b007c4559e1877030524e9e6ec49fafc7eb94d81611aa2d89cd5f5" exitCode=0
Oct 29 23:17:03.218185 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:03.217788    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw" event={"ID":"d47b2c1f-2018-4906-bd8c-b44ee9b8240b","Type":"ContainerDied","Data":"858d413e40b007c4559e1877030524e9e6ec49fafc7eb94d81611aa2d89cd5f5"}
Oct 29 23:17:04.224869 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:04.224833    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw" event={"ID":"d47b2c1f-2018-4906-bd8c-b44ee9b8240b","Type":"ContainerStarted","Data":"b83c008847b5cf5efeb23e81322ad6fae4b3fea8439cd0b2027d2fbb8f437eaa"}
Oct 29 23:17:04.225389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:04.224898    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:17:04.309764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:04.309716    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw" podStartSLOduration=1.677304221 podStartE2EDuration="5.30970065s" totalImagesPullingTime="3.1391756s" totalInitContainerRuntime="493.220829ms" isStatefulPod=false podCreationTimestamp="2026-10-29 23:16:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:17:04.296855957 +0000 UTC m=+235.383139091" watchObservedRunningTime="2026-10-29 23:17:04.30970065 +0000 UTC m=+235.395983786"
Oct 29 23:17:07.855079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:07.855045    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:08.262006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:08.261915    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:15.231361 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:15.231328    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bb7fb9d97-zd2fw"
Oct 29 23:17:16.046994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:16.046949    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:16.207639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:16.207604    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:18.724935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:18.724901    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:18.883807 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:18.883769    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:19.033994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.033885    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:19.190387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.190356    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"]
Oct 29 23:17:19.198014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.197990    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.213478 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.213453    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"]
Oct 29 23:17:19.251600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.251574    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:19.302113 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302028    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tmp\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.302113 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302073    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w28lj\" (UniqueName: \"kubernetes.io/projected/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-kube-api-access-w28lj\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.302290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302207    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tls-certs\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.302730 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302712    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 29 23:17:19.302730 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302732    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 29 23:17:19.302856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.302767    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-m5slx\""
Oct 29 23:17:19.403584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.403546    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tmp\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.403584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.403587    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w28lj\" (UniqueName: \"kubernetes.io/projected/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-kube-api-access-w28lj\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.403804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.403652    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tls-certs\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.406025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.405997    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tmp\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.406138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.406106    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-tls-certs\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.423538 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.423510    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w28lj\" (UniqueName: \"kubernetes.io/projected/3b3ff99f-81d0-4bc1-b47a-ebb26e736c02-kube-api-access-w28lj\") pod \"kube-auth-proxy-7cbcb88b9-g7xvv\" (UID: \"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02\") " pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.484433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.484395    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:19.512920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.512885    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"
Oct 29 23:17:19.801755 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:19.801719    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv"]
Oct 29 23:17:19.802800 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:17:19.802775    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b3ff99f_81d0_4bc1_b47a_ebb26e736c02.slice/crio-1540a9162c3a9107419ec5e15f0a828e9456a138f0db43a1df9fbfe6b302c347 WatchSource:0}: Error finding container 1540a9162c3a9107419ec5e15f0a828e9456a138f0db43a1df9fbfe6b302c347: Status 404 returned error can't find the container with id 1540a9162c3a9107419ec5e15f0a828e9456a138f0db43a1df9fbfe6b302c347
Oct 29 23:17:20.288785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:20.288739    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv" event={"ID":"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02","Type":"ContainerStarted","Data":"1540a9162c3a9107419ec5e15f0a828e9456a138f0db43a1df9fbfe6b302c347"}
Oct 29 23:17:20.697951 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:20.697913    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:20.855160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:20.855069    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:21.774575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:21.774532    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:22.299296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.299258    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv" event={"ID":"3b3ff99f-81d0-4bc1-b47a-ebb26e736c02","Type":"ContainerStarted","Data":"2c6b7ef8adb7193585f29b5b8c507f82fcabfd539ba5aee387e6c46835bba96c"}
Oct 29 23:17:22.341501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.341447    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cbcb88b9-g7xvv" podStartSLOduration=0.923517486 podStartE2EDuration="3.341434458s" totalImagesPullingTime="2.417916972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:17:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:17:22.340039613 +0000 UTC m=+253.426322747" watchObservedRunningTime="2026-10-29 23:17:22.341434458 +0000 UTC m=+253.427717593"
Oct 29 23:17:22.495046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.494944    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:22.711261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.711229    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-x4vpd"]
Oct 29 23:17:22.714640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.714616    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:22.732593 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.732552    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\""
Oct 29 23:17:22.733819 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.733792    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-x4vpd"]
Oct 29 23:17:22.744087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.744064    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-z5dhj\""
Oct 29 23:17:22.804346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.804272    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:22.833149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.833118    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:22.833282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.833161    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78b9f\" (UniqueName: \"kubernetes.io/projected/02a5137b-4531-4484-9661-6257be71c966-kube-api-access-78b9f\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:22.933917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.933880    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:22.933917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.933920    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78b9f\" (UniqueName: \"kubernetes.io/projected/02a5137b-4531-4484-9661-6257be71c966-kube-api-access-78b9f\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:22.934152 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:17:22.934059    5111 secret.go:189] Couldn't get secret opendatahub/odh-observability-webhook-cert: secret "odh-observability-webhook-cert" not found
Oct 29 23:17:22.934152 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:17:22.934119    5111 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs podName:02a5137b-4531-4484-9661-6257be71c966 nodeName:}" failed. No retries permitted until 2026-10-29 23:17:23.43410337 +0000 UTC m=+254.520386499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs") pod "odh-observability-584f99c788-x4vpd" (UID: "02a5137b-4531-4484-9661-6257be71c966") : secret "odh-observability-webhook-cert" not found
Oct 29 23:17:22.953492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:22.953469    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78b9f\" (UniqueName: \"kubernetes.io/projected/02a5137b-4531-4484-9661-6257be71c966-kube-api-access-78b9f\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:23.181742 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:23.181708    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:23.438970 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:23.438884    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:23.441412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:23.441389    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/02a5137b-4531-4484-9661-6257be71c966-webhook-certs\") pod \"odh-observability-584f99c788-x4vpd\" (UID: \"02a5137b-4531-4484-9661-6257be71c966\") " pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:23.626814 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:23.626779    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:23.920188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:23.920164    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-x4vpd"]
Oct 29 23:17:23.921161 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:17:23.921139    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a5137b_4531_4484_9661_6257be71c966.slice/crio-75854c6776d44e6f0aab0df49a3c260fb38e6a4860b8a869e7a54f0b76f6672b WatchSource:0}: Error finding container 75854c6776d44e6f0aab0df49a3c260fb38e6a4860b8a869e7a54f0b76f6672b: Status 404 returned error can't find the container with id 75854c6776d44e6f0aab0df49a3c260fb38e6a4860b8a869e7a54f0b76f6672b
Oct 29 23:17:24.307966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:24.307888    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-x4vpd" event={"ID":"02a5137b-4531-4484-9661-6257be71c966","Type":"ContainerStarted","Data":"75854c6776d44e6f0aab0df49a3c260fb38e6a4860b8a869e7a54f0b76f6672b"}
Oct 29 23:17:25.277531 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:25.277496    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:25.436407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:25.436361    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:26.003514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:26.003477    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:26.144291 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:26.144258    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:26.149500 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:17:26.149468    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:17:26.392633 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:26.392579    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:27.325671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:27.325639    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-x4vpd" event={"ID":"02a5137b-4531-4484-9661-6257be71c966","Type":"ContainerStarted","Data":"41fa63ecb3ea65237a2176fcc4472f6f31d7518774ff74a9814871a2590a456a"}
Oct 29 23:17:27.325942 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:27.325773    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:27.367423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:27.367377    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-x4vpd" podStartSLOduration=2.804819147 podStartE2EDuration="5.36736249s" totalImagesPullingTime="2.562543343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:17:27.366588239 +0000 UTC m=+258.452871373" watchObservedRunningTime="2026-10-29 23:17:27.36736249 +0000 UTC m=+258.453645624"
Oct 29 23:17:36.302819 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.302756    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:36.334144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.334113    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"]
Oct 29 23:17:36.337241 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.337226    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.361162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.361135    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"]
Oct 29 23:17:36.395229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.395197    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\""
Oct 29 23:17:36.395229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.395219    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-s68qv\""
Oct 29 23:17:36.451662 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.451630    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.451820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.451669    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8440b63-eab9-4f18-99b5-6746c354b613-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.451820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.451694    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.451889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.451809    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77jj\" (UniqueName: \"kubernetes.io/projected/c8440b63-eab9-4f18-99b5-6746c354b613-kube-api-access-d77jj\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.463325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.463297    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:36.552459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.552424    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77jj\" (UniqueName: \"kubernetes.io/projected/c8440b63-eab9-4f18-99b5-6746c354b613-kube-api-access-d77jj\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.552617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.552485    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.552617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.552511    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8440b63-eab9-4f18-99b5-6746c354b613-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.552617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.552534    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.552906 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.552868    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-tmp\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.553055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.553032    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c8440b63-eab9-4f18-99b5-6746c354b613-manifests\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.553685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.553654    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c8440b63-eab9-4f18-99b5-6746c354b613-trusted-ca\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.571931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.571904    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77jj\" (UniqueName: \"kubernetes.io/projected/c8440b63-eab9-4f18-99b5-6746c354b613-kube-api-access-d77jj\") pod \"kserve-module-controller-manager-5cfb77bf8c-qzzlh\" (UID: \"c8440b63-eab9-4f18-99b5-6746c354b613\") " pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.616263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.616236    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:17:36.651271 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.651244    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:36.961304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:36.955311    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"]
Oct 29 23:17:37.367557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:37.367518    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh" event={"ID":"c8440b63-eab9-4f18-99b5-6746c354b613","Type":"ContainerStarted","Data":"1d67066d32911c6bccf8555f4b0e4a595a9d09ea321438e3fddb21d0aa36e02e"}
Oct 29 23:17:38.332707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:38.332668    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-x4vpd"
Oct 29 23:17:39.378236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:39.378200    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh" event={"ID":"c8440b63-eab9-4f18-99b5-6746c354b613","Type":"ContainerStarted","Data":"1a6310f7aff45232adc9dcbc6cb7651e459aa244287aace8338c69b340323b4d"}
Oct 29 23:17:39.378650 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:39.378316    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:17:39.431305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:39.431255    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh" podStartSLOduration=1.8843753840000002 podStartE2EDuration="3.431238978s" totalImagesPullingTime="1.546863594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:17:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:17:39.430684881 +0000 UTC m=+270.516968014" watchObservedRunningTime="2026-10-29 23:17:39.431238978 +0000 UTC m=+270.517522115"
Oct 29 23:17:56.147117 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:17:56.147082    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:17:58.507770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:17:58.507735    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:00.383703 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:00.383672    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-5cfb77bf8c-qzzlh"
Oct 29 23:18:09.072984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:09.072933    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:18:09.073849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:09.073828    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:18:22.423357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.423267    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:22.593341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.593308    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qssv2"]
Oct 29 23:18:22.596857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.596836    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.632436 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.632409    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-d746s\""
Oct 29 23:18:22.644715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.644685    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 29 23:18:22.645949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.645930    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qssv2"]
Oct 29 23:18:22.647769 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.647750    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:18:22.651090 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.651072    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.655207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.655177    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75s6r\" (UniqueName: \"kubernetes.io/projected/d4c958e6-6b39-48e0-be3c-2362004a307b-kube-api-access-75s6r\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.655302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.655240    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4c958e6-6b39-48e0-be3c-2362004a307b-cert\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.669711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.669688    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:18:22.675201 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.675155    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qwkxl"]
Oct 29 23:18:22.678754 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.678732    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:22.678846 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.678834    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.696500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.696479    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qwkxl"]
Oct 29 23:18:22.714509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.714486    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-n5rdp"]
Oct 29 23:18:22.718201 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.718183    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.731427 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.731408    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cl8nt\""
Oct 29 23:18:22.731511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.731462    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 29 23:18:22.731511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.731488    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 29 23:18:22.731603 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.731511    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-nx8q9\""
Oct 29 23:18:22.735112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.735093    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-n5rdp"]
Oct 29 23:18:22.746940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.746922    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 29 23:18:22.747061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.746955    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9zvfj\""
Oct 29 23:18:22.747061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.747049    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 29 23:18:22.756080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756058    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4c958e6-6b39-48e0-be3c-2362004a307b-cert\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.756168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756088    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.756168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756113    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a0d6f6-5f87-422d-b41e-ed69859aa225-tls-certs\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.756168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756131    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkxsf\" (UniqueName: \"kubernetes.io/projected/28a0d6f6-5f87-422d-b41e-ed69859aa225-kube-api-access-nkxsf\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.756355 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756333    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shfkz\" (UniqueName: \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.756419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.756402    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75s6r\" (UniqueName: \"kubernetes.io/projected/d4c958e6-6b39-48e0-be3c-2362004a307b-kube-api-access-75s6r\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.758784 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.758760    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d4c958e6-6b39-48e0-be3c-2362004a307b-cert\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.775805 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.775784    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75s6r\" (UniqueName: \"kubernetes.io/projected/d4c958e6-6b39-48e0-be3c-2362004a307b-kube-api-access-75s6r\") pod \"kserve-controller-manager-776cc66d47-qssv2\" (UID: \"d4c958e6-6b39-48e0-be3c-2362004a307b\") " pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.823297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.823258    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:22.857024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.856990    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-cert\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.857155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857058    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shfkz\" (UniqueName: \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.857155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857122    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qfsp\" (UniqueName: \"kubernetes.io/projected/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-kube-api-access-6qfsp\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.857263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857206    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.857314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857279    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a0d6f6-5f87-422d-b41e-ed69859aa225-tls-certs\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.857364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857306    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkxsf\" (UniqueName: \"kubernetes.io/projected/28a0d6f6-5f87-422d-b41e-ed69859aa225-kube-api-access-nkxsf\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.857364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.857346    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-metrics-certs\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.859707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.859683    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.859881 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.859866    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28a0d6f6-5f87-422d-b41e-ed69859aa225-tls-certs\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.876486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.876461    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shfkz\" (UniqueName: \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\") pod \"llmisvc-controller-manager-869d5dcc5d-tksvc\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.876694 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.876676    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkxsf\" (UniqueName: \"kubernetes.io/projected/28a0d6f6-5f87-422d-b41e-ed69859aa225-kube-api-access-nkxsf\") pod \"model-serving-api-75bd89fdfc-qwkxl\" (UID: \"28a0d6f6-5f87-422d-b41e-ed69859aa225\") " pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:22.909858 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.909838    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:22.958158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.958114    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-metrics-certs\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.958291 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.958180    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-cert\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.958291 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.958245    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qfsp\" (UniqueName: \"kubernetes.io/projected/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-kube-api-access-6qfsp\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.960690 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.960648    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:22.961754 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.961727    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-metrics-certs\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.963010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.962955    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-cert\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.963138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.963121    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:22.982182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.982159    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qfsp\" (UniqueName: \"kubernetes.io/projected/9c40a8fd-7e19-48e9-8f5d-8f43c798d235-kube-api-access-6qfsp\") pod \"odh-model-controller-f87fcf56c-n5rdp\" (UID: \"9c40a8fd-7e19-48e9-8f5d-8f43c798d235\") " pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:22.991131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:22.991109    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:23.030437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.030385    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:23.109342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.109308    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:23.206866 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.206835    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-qssv2"]
Oct 29 23:18:23.207224 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:18:23.207178    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c958e6_6b39_48e0_be3c_2362004a307b.slice/crio-68f55664e43518635a6c9bba4faec4a2cd3bd39e0f513c77b5692267d6b677d7 WatchSource:0}: Error finding container 68f55664e43518635a6c9bba4faec4a2cd3bd39e0f513c77b5692267d6b677d7: Status 404 returned error can't find the container with id 68f55664e43518635a6c9bba4faec4a2cd3bd39e0f513c77b5692267d6b677d7
Oct 29 23:18:23.208508 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.208492    5111 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:18:23.271281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.271254    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:18:23.271377 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:18:23.271352    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda378f5e_3ad2_4227_90fc_027c999a70e6.slice/crio-d84da1fb1f77eb33bb4ee21a222fb52850558e874f563d1a649521b97a0efd64 WatchSource:0}: Error finding container d84da1fb1f77eb33bb4ee21a222fb52850558e874f563d1a649521b97a0efd64: Status 404 returned error can't find the container with id d84da1fb1f77eb33bb4ee21a222fb52850558e874f563d1a649521b97a0efd64
Oct 29 23:18:23.294838 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.294816    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-qwkxl"]
Oct 29 23:18:23.294929 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:18:23.294915    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a0d6f6_5f87_422d_b41e_ed69859aa225.slice/crio-305b181768895d7fed5ebcf97c5f48d60ca01f7a7393c74ce89a6a097e95041d WatchSource:0}: Error finding container 305b181768895d7fed5ebcf97c5f48d60ca01f7a7393c74ce89a6a097e95041d: Status 404 returned error can't find the container with id 305b181768895d7fed5ebcf97c5f48d60ca01f7a7393c74ce89a6a097e95041d
Oct 29 23:18:23.341230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.341204    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-n5rdp"]
Oct 29 23:18:23.342880 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:18:23.342858    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c40a8fd_7e19_48e9_8f5d_8f43c798d235.slice/crio-65c10aa2a7014d202e84bfcc6edd4e06477ba192b66fca10c500dc298f6acdce WatchSource:0}: Error finding container 65c10aa2a7014d202e84bfcc6edd4e06477ba192b66fca10c500dc298f6acdce: Status 404 returned error can't find the container with id 65c10aa2a7014d202e84bfcc6edd4e06477ba192b66fca10c500dc298f6acdce
Oct 29 23:18:23.563116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.563031    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" event={"ID":"da378f5e-3ad2-4227-90fc-027c999a70e6","Type":"ContainerStarted","Data":"d84da1fb1f77eb33bb4ee21a222fb52850558e874f563d1a649521b97a0efd64"}
Oct 29 23:18:23.564079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.564058    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp" event={"ID":"9c40a8fd-7e19-48e9-8f5d-8f43c798d235","Type":"ContainerStarted","Data":"65c10aa2a7014d202e84bfcc6edd4e06477ba192b66fca10c500dc298f6acdce"}
Oct 29 23:18:23.565215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.565193    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl" event={"ID":"28a0d6f6-5f87-422d-b41e-ed69859aa225","Type":"ContainerStarted","Data":"305b181768895d7fed5ebcf97c5f48d60ca01f7a7393c74ce89a6a097e95041d"}
Oct 29 23:18:23.566431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:23.566412    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2" event={"ID":"d4c958e6-6b39-48e0-be3c-2362004a307b","Type":"ContainerStarted","Data":"68f55664e43518635a6c9bba4faec4a2cd3bd39e0f513c77b5692267d6b677d7"}
Oct 29 23:18:24.798519 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:24.798482    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:24.938539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:24.938503    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:25.085155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:25.085119    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:25.222609 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:25.222565    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:18:26.158209 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:18:26.158136    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:18:26.590959 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.590921    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2" event={"ID":"d4c958e6-6b39-48e0-be3c-2362004a307b","Type":"ContainerStarted","Data":"dc964f2a3d8ed40341f3e168bc492c5e4f82d000fe6a1d6709f564d6f1b0821e"}
Oct 29 23:18:26.591163 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.591010    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:26.593124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.593090    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" event={"ID":"da378f5e-3ad2-4227-90fc-027c999a70e6","Type":"ContainerStarted","Data":"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"}
Oct 29 23:18:26.593268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.593188    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:26.594821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.594797    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl" event={"ID":"28a0d6f6-5f87-422d-b41e-ed69859aa225","Type":"ContainerStarted","Data":"cc03162a2b821ee0f80b246c67d2d9e4ef812f1d7402393407c79dc63ac7cb3d"}
Oct 29 23:18:26.594952 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.594921    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:26.630380 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.630278    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2" podStartSLOduration=1.541780765 podStartE2EDuration="4.630259782s" totalImagesPullingTime="3.088479017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:26.629218623 +0000 UTC m=+317.715501758" watchObservedRunningTime="2026-10-29 23:18:26.630259782 +0000 UTC m=+317.716542918"
Oct 29 23:18:26.709676 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.709623    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl" podStartSLOduration=1.885679962 podStartE2EDuration="4.709605951s" totalImagesPullingTime="2.823925989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:26.708334333 +0000 UTC m=+317.794617467" watchObservedRunningTime="2026-10-29 23:18:26.709605951 +0000 UTC m=+317.795889084"
Oct 29 23:18:26.710339 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:26.710293    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" podStartSLOduration=1.630398729 podStartE2EDuration="4.71027735s" totalImagesPullingTime="3.079878621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:26.669363098 +0000 UTC m=+317.755646233" watchObservedRunningTime="2026-10-29 23:18:26.71027735 +0000 UTC m=+317.796560486"
Oct 29 23:18:27.600597 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:27.600552    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp" event={"ID":"9c40a8fd-7e19-48e9-8f5d-8f43c798d235","Type":"ContainerStarted","Data":"5f30eeac6100b958de22b8256f8dc7e86184fdd1db3e9cd135e0cc1ff884de1c"}
Oct 29 23:18:27.601068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:27.600646    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:27.640704 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:27.640654    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp" podStartSLOduration=2.048863697 podStartE2EDuration="5.640641468s" totalImagesPullingTime="3.591777771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:27.640141249 +0000 UTC m=+318.726424384" watchObservedRunningTime="2026-10-29 23:18:27.640641468 +0000 UTC m=+318.726924601"
Oct 29 23:18:35.862025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.861989    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"]
Oct 29 23:18:35.866523 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.866504    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:35.886596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.886558    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"]
Oct 29 23:18:35.914312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.914278    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 29 23:18:35.916655 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.916640    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 29 23:18:35.916721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.916707    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 29 23:18:35.974310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.974284    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 29 23:18:35.974310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.974286    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 29 23:18:35.974310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.974286    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-gwcll\""
Oct 29 23:18:35.978572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.978551    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-manager-config\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:35.978646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.978582    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-metrics-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:35.978721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.978703    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7bvj\" (UniqueName: \"kubernetes.io/projected/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-kube-api-access-k7bvj\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:35.978757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:35.978746    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.079395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.079354    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7bvj\" (UniqueName: \"kubernetes.io/projected/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-kube-api-access-k7bvj\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.079583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.079411    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.079583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.079444    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-manager-config\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.079583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.079464    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-metrics-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.080171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.080144    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-manager-config\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.081959 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.081939    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-metrics-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.082064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.082024    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-cert\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.098642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.098614    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7bvj\" (UniqueName: \"kubernetes.io/projected/57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a-kube-api-access-k7bvj\") pod \"lws-controller-manager-6bfdd46644-rbg6b\" (UID: \"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a\") " pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.184654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.184576    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:36.483335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.483308    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"]
Oct 29 23:18:36.484009 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:18:36.483967    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57a5f1e5_4e8a_46f0_b7ab_69ecb78b9b5a.slice/crio-8f056ef8b9ef55495e18669a1b952b4760bbb3291a0e0c39d4343e8b41d246a0 WatchSource:0}: Error finding container 8f056ef8b9ef55495e18669a1b952b4760bbb3291a0e0c39d4343e8b41d246a0: Status 404 returned error can't find the container with id 8f056ef8b9ef55495e18669a1b952b4760bbb3291a0e0c39d4343e8b41d246a0
Oct 29 23:18:36.640469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:36.640428    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b" event={"ID":"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a","Type":"ContainerStarted","Data":"8f056ef8b9ef55495e18669a1b952b4760bbb3291a0e0c39d4343e8b41d246a0"}
Oct 29 23:18:37.605938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:37.605909    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-qwkxl"
Oct 29 23:18:38.608006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:38.607962    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-n5rdp"
Oct 29 23:18:39.654630 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:39.654598    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b" event={"ID":"57a5f1e5-4e8a-46f0-b7ab-69ecb78b9b5a","Type":"ContainerStarted","Data":"ad9e0769d8d3e572d545cbe7e1675d570b36326ff5405a296c0d9be393bd902e"}
Oct 29 23:18:39.655114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:39.654680    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:39.696592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:39.696539    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b" podStartSLOduration=2.494231918 podStartE2EDuration="4.696526648s" totalImagesPullingTime="2.20229473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:18:39.694827348 +0000 UTC m=+330.781110481" watchObservedRunningTime="2026-10-29 23:18:39.696526648 +0000 UTC m=+330.782809843"
Oct 29 23:18:50.661131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:50.661098    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bfdd46644-rbg6b"
Oct 29 23:18:56.145668 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:18:56.145616    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:18:57.602645 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:57.602616    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:18:57.606265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:57.606242    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-qssv2"
Oct 29 23:18:58.620175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:18:58.620130    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:12.020171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:12.020135    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:12.169590 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:12.169561    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:26.148086 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:19:26.148050    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:19:28.067420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:28.067382    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:28.830388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:28.830333    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:29.625091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:29.625056    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:32.807164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:32.807129    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:33.241648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:33.241619    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:35.095766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:35.095725    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:35.280844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:35.280810    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:36.461039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:36.461005    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:37.223725 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:37.223693    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:37.696401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:37.696361    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:38.736794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:38.736760    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:38.932423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:38.932394    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:40.449265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:40.449209    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:40.623047 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:40.623013    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:43.219178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:43.219149    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:43.368167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:43.368128    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:43.620493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:43.620466    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:43.757625 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:43.757598    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:44.022133 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:44.022061    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:44.163146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:44.163121    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:49.917035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:49.917001    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:50.154556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:50.154490    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:54.467591 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:54.467513    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:54.614839 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:54.614808    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:56.150047 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:19:56.150009    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:19:58.731277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:58.731243    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:59.246547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:59.246512    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:19:59.386021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:19:59.385970    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:04.447237 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:04.447203    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:04.606746 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:04.606713    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:04.842815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:04.842785    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:05.004991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:05.004951    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:05.218169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:05.218100    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:11.095219 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:11.095185    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:11.323892 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:11.323836    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:11.596968 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:11.596935    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:11.781534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:11.781501    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:16.253795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:16.253761    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:16.454182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:16.454150    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:18.111996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:18.111952    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:21.079787 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:21.079742    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:21.227573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:21.227543    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:23.494584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:23.494546    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:23.655229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:23.655190    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:26.148296 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:20:26.148252    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:20:37.004811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:37.004767    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:37.174328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:37.174297    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:37.925888 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:37.925855    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:38.072554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:38.072519    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:56.148116 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:20:56.148081    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:20:57.849514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:57.849476    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:57.999669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:57.999639    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:58.608472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:58.608435    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:20:58.846608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:20:58.846573    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:10.951643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:10.951607    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:13.089536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:13.089499    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:20.678124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:20.678047    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:20.822445 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:20.822416    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:20.974507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:20.974445    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:21.115309 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:21.115277    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:24.308707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:24.308675    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:24.463662 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:24.463630    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:26.148777 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:21:26.148741    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:21:27.784399 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:27.784360    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:27.944713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:27.944676    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:40.459258 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:40.459224    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:47.346939 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:47.346904    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:56.148384 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:21:56.148346    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:21:57.144907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:57.144873    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:57.289678 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:57.289650    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:58.600524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:58.600487    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:58.744218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:58.744182    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:58.890490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:58.890414    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:21:59.039437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:21:59.039405    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:04.157490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:04.157447    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:04.301371 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:04.301326    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:07.302722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:07.302686    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:08.866824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:08.866787    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"]
Oct 29 23:22:08.870357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:08.870336    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:08.889669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:08.889646    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"]
Oct 29 23:22:08.925466 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:08.925441    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hcsm\" (UniqueName: \"kubernetes.io/projected/f31401ea-48ef-4ffa-a946-eb312302cbbd-kube-api-access-9hcsm\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:08.925567 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:08.925548    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f31401ea-48ef-4ffa-a946-eb312302cbbd-cert\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.026389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.026365    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hcsm\" (UniqueName: \"kubernetes.io/projected/f31401ea-48ef-4ffa-a946-eb312302cbbd-kube-api-access-9hcsm\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.026505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.026425    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f31401ea-48ef-4ffa-a946-eb312302cbbd-cert\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.028751 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.028726    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f31401ea-48ef-4ffa-a946-eb312302cbbd-cert\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.050649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.050630    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hcsm\" (UniqueName: \"kubernetes.io/projected/f31401ea-48ef-4ffa-a946-eb312302cbbd-kube-api-access-9hcsm\") pod \"llmisvc-controller-manager-6466c554d8-96hcn\" (UID: \"f31401ea-48ef-4ffa-a946-eb312302cbbd\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.183670 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.183613    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:09.477611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.477587    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"]
Oct 29 23:22:09.480087 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:22:09.480061    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf31401ea_48ef_4ffa_a946_eb312302cbbd.slice/crio-d9c1c8f55650f4faf219a208589df852bc0bcbbe7a905f3fe26497709d46821d WatchSource:0}: Error finding container d9c1c8f55650f4faf219a208589df852bc0bcbbe7a905f3fe26497709d46821d: Status 404 returned error can't find the container with id d9c1c8f55650f4faf219a208589df852bc0bcbbe7a905f3fe26497709d46821d
Oct 29 23:22:09.569938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:09.569901    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn" event={"ID":"f31401ea-48ef-4ffa-a946-eb312302cbbd","Type":"ContainerStarted","Data":"d9c1c8f55650f4faf219a208589df852bc0bcbbe7a905f3fe26497709d46821d"}
Oct 29 23:22:10.576398 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:10.576362    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn" event={"ID":"f31401ea-48ef-4ffa-a946-eb312302cbbd","Type":"ContainerStarted","Data":"83c1caa2d9440e01ca3031a8e451ae51b785edd652c2679b0ebcd905bb03f526"}
Oct 29 23:22:10.576850 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:10.576431    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:10.621229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:10.621183    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn" podStartSLOduration=2.381335367 podStartE2EDuration="2.621170904s" totalImagesPullingTime="239.835537ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:10.619938404 +0000 UTC m=+541.706221537" watchObservedRunningTime="2026-10-29 23:22:10.621170904 +0000 UTC m=+541.707454017"
Oct 29 23:22:26.148208 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:22:26.148170    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:22:41.581718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.581684    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-96hcn"
Oct 29 23:22:41.649234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.649206    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:22:41.649433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.649388    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" podUID="da378f5e-3ad2-4227-90fc-027c999a70e6" containerName="manager" containerID="cri-o://0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393" gracePeriod=30
Oct 29 23:22:41.894944 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.894917    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:22:41.975382 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.975354    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\" (UniqueName: \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\") pod \"da378f5e-3ad2-4227-90fc-027c999a70e6\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") "
Oct 29 23:22:41.975506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.975442    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\" (UniqueName: \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\") pod \"da378f5e-3ad2-4227-90fc-027c999a70e6\" (UID: \"da378f5e-3ad2-4227-90fc-027c999a70e6\") "
Oct 29 23:22:41.977468 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.977443    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz" pod "da378f5e-3ad2-4227-90fc-027c999a70e6" (UID: "da378f5e-3ad2-4227-90fc-027c999a70e6"). InnerVolumeSpecName "kube-api-access-shfkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:22:41.977536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:41.977479    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert" pod "da378f5e-3ad2-4227-90fc-027c999a70e6" (UID: "da378f5e-3ad2-4227-90fc-027c999a70e6"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:22:42.076276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.076250    5111 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da378f5e-3ad2-4227-90fc-027c999a70e6-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:22:42.076276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.076275    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shfkz\" (UniqueName: \"kubernetes.io/projected/da378f5e-3ad2-4227-90fc-027c999a70e6-kube-api-access-shfkz\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:22:42.519512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.519485    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:42.725717 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.725684    5111 generic.go:356] "Generic (PLEG): container finished" podID="da378f5e-3ad2-4227-90fc-027c999a70e6" containerID="0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393" exitCode=0
Oct 29 23:22:42.726120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.725734    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"
Oct 29 23:22:42.726120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.725739    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" event={"ID":"da378f5e-3ad2-4227-90fc-027c999a70e6","Type":"ContainerDied","Data":"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"}
Oct 29 23:22:42.726120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.725841    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc" event={"ID":"da378f5e-3ad2-4227-90fc-027c999a70e6","Type":"ContainerDied","Data":"d84da1fb1f77eb33bb4ee21a222fb52850558e874f563d1a649521b97a0efd64"}
Oct 29 23:22:42.726120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.725865    5111 scope.go:122] "RemoveContainer" containerID="0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"
Oct 29 23:22:42.734877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.734861    5111 scope.go:122] "RemoveContainer" containerID="0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"
Oct 29 23:22:42.735154 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:22:42.735134    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393\": container with ID starting with 0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393 not found: ID does not exist" containerID="0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"
Oct 29 23:22:42.735254 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.735165    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393"} err="failed to get container status \"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393\": rpc error: code = NotFound desc = could not find container \"0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393\": container with ID starting with 0221a667dc6ac127ed3ba0987527230e623720adc25b15c01e74d51a3ca66393 not found: ID does not exist"
Oct 29 23:22:42.745680 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.745656    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:22:42.791834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.791783    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:22:42.805533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:42.805510    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-tksvc"]
Oct 29 23:22:43.059633 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:43.059560    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da378f5e-3ad2-4227-90fc-027c999a70e6" path="/var/lib/kubelet/pods/da378f5e-3ad2-4227-90fc-027c999a70e6/volumes"
Oct 29 23:22:56.148708 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:22:56.148625    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:22:59.066591 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:22:59.066556    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:00.120938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:00.120901    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:02.023087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:02.023013    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:03.187300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:03.187264    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:06.007749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:06.007709    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:07.404715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:07.404667    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:09.103369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:09.103346    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:23:09.105085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:09.105064    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:23:10.099277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:10.099239    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:11.717921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:11.717885    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:25.868016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:25.867982    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:26.005168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:26.005125    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:26.147723 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:23:26.147654    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:23:29.925293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:29.925265    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:30.084015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:30.083967    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:39.063459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.063423    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"]
Oct 29 23:23:39.064272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.064248    5111 cpu_manager.go:414] "containerMap: removing container" podUID="da378f5e-3ad2-4227-90fc-027c999a70e6" containerName="manager"
Oct 29 23:23:39.064272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.064272    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da378f5e-3ad2-4227-90fc-027c999a70e6" containerName="manager"
Oct 29 23:23:39.064506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.064493    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="da378f5e-3ad2-4227-90fc-027c999a70e6" containerName="manager"
Oct 29 23:23:39.068623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.068599    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.086809 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.086780    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"]
Oct 29 23:23:39.128204 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128180    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128226    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128270    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lglbt\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-kube-api-access-lglbt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128393 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128323    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128393 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128344    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128439    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128595 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128578    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128622    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.128726 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.128714    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.169122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.169103    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 29 23:23:39.169260 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.169244    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-tvvvz\""
Oct 29 23:23:39.229740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229718    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229759    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229782    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lglbt\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-kube-api-access-lglbt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229826    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229858    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229903    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.229949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229935    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230125 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.229957    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230125 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230000    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230197    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230258    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230435    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230537 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230519    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.230670 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.230652    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.232194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.232169    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.232305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.232213    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.249844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.249823    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.249953 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.249826    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lglbt\" (UniqueName: \"kubernetes.io/projected/d1c4d693-e49e-41d5-935c-3e12cd99e3b5-kube-api-access-lglbt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-wpwhs\" (UID: \"d1c4d693-e49e-41d5-935c-3e12cd99e3b5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.382766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.382743    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:39.696548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.696526    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"]
Oct 29 23:23:39.699324 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:23:39.699280    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1c4d693_e49e_41d5_935c_3e12cd99e3b5.slice/crio-dd06998672e3b4cb74c652a28e6ace144e95e3425237e7d80a8d8ffffb2b4eff WatchSource:0}: Error finding container dd06998672e3b4cb74c652a28e6ace144e95e3425237e7d80a8d8ffffb2b4eff: Status 404 returned error can't find the container with id dd06998672e3b4cb74c652a28e6ace144e95e3425237e7d80a8d8ffffb2b4eff
Oct 29 23:23:39.701303 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.701286    5111 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 29 23:23:39.991132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:39.991054    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" event={"ID":"d1c4d693-e49e-41d5-935c-3e12cd99e3b5","Type":"ContainerStarted","Data":"dd06998672e3b4cb74c652a28e6ace144e95e3425237e7d80a8d8ffffb2b4eff"}
Oct 29 23:23:42.399273 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:42.399243    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:42.399543 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:42.399318    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:42.399543 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:42.399348    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:43.007346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:43.007309    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" event={"ID":"d1c4d693-e49e-41d5-935c-3e12cd99e3b5","Type":"ContainerStarted","Data":"26534128216b2234d96b2119c1bf6dccd5898df67d05ee40e5fb6bcfd4df98eb"}
Oct 29 23:23:43.397238 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:43.397194    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:43.398836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:43.398808    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:43.398997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:43.398864    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:44.394000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:44.392900    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:44.394000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:44.392963    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:45.391355 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:45.391317    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:45.391602 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:45.391572    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:46.387328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:46.387291    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:46.387947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:46.387907    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:47.389998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:47.386314    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:47.389998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:47.386383    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:48.383369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:48.383335    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:48.383535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:48.383412    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:49.384139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:49.384108    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Oct 29 23:23:49.384522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:49.384155    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Oct 29 23:23:50.593415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:50.593379    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:50.594069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:50.594043    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:50.594760 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:50.594740    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs"
Oct 29 23:23:50.664609 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:50.664568    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podStartSLOduration=8.96757623 podStartE2EDuration="11.664558136s" totalImagesPullingTime="2.696981906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:43.09026201 +0000 UTC m=+634.176545145" watchObservedRunningTime="2026-10-29 23:23:50.664558136 +0000 UTC m=+641.750841334"
Oct 29 23:23:54.113237 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.113205    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"]
Oct 29 23:23:54.116553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.116537    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.148791 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.148748    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"]
Oct 29 23:23:54.180058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180030    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180062    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47st7\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-kube-api-access-47st7\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180088    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180170    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180200    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180220    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180241    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180324    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0eab29-57b8-4012-8f45-04a0914fcd73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.180437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.180348    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281438 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281405    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281549 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281465    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281549 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281489    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281549 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281518    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281567    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0eab29-57b8-4012-8f45-04a0914fcd73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281593    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281656    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281682    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47st7\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-kube-api-access-47st7\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281885 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281718    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.281946 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.281881    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.282134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.282109    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.282243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.282220    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.282373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.282285    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.282505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.282479    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed0eab29-57b8-4012-8f45-04a0914fcd73-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.283846 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.283823    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.284588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.284567    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.302499 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.302471    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.302621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.302597    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47st7\" (UniqueName: \"kubernetes.io/projected/ed0eab29-57b8-4012-8f45-04a0914fcd73-kube-api-access-47st7\") pod \"maas-default-gateway-openshift-default-746ff8f49f-gdsfs\" (UID: \"ed0eab29-57b8-4012-8f45-04a0914fcd73\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.429625 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.429563    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:54.742375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.742348    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"]
Oct 29 23:23:54.743349 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:23:54.743325    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded0eab29_57b8_4012_8f45_04a0914fcd73.slice/crio-d90abc2700a4267284a57bc7cfd14a2d7f1a511c7b785183f47fb9c24fd63b5b WatchSource:0}: Error finding container d90abc2700a4267284a57bc7cfd14a2d7f1a511c7b785183f47fb9c24fd63b5b: Status 404 returned error can't find the container with id d90abc2700a4267284a57bc7cfd14a2d7f1a511c7b785183f47fb9c24fd63b5b
Oct 29 23:23:54.746203 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.746173    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:54.746278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.746238    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:54.746278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:54.746266    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:23:55.067922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.067842    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" event={"ID":"ed0eab29-57b8-4012-8f45-04a0914fcd73","Type":"ContainerStarted","Data":"d133d9a0a5cd957968e49e86198241a84adea82a2484f204704d051e2fcf2cc3"}
Oct 29 23:23:55.067922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.067895    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" event={"ID":"ed0eab29-57b8-4012-8f45-04a0914fcd73","Type":"ContainerStarted","Data":"d90abc2700a4267284a57bc7cfd14a2d7f1a511c7b785183f47fb9c24fd63b5b"}
Oct 29 23:23:55.127286 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.127238    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podStartSLOduration=1.127224313 podStartE2EDuration="1.127224313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:55.126548416 +0000 UTC m=+646.212831549" watchObservedRunningTime="2026-10-29 23:23:55.127224313 +0000 UTC m=+646.213507444"
Oct 29 23:23:55.430586 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.430552    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:23:55.431880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.431857    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:23:55.431935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.431906    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:23:55.864344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:55.864306    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:56.147100 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:23:56.147014    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:23:56.371243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:56.371207    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:56.430136 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:56.430071    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:23:56.430136 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:56.430128    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:23:57.430680 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:57.430648    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:23:57.431084 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:57.430706    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:23:58.010802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:58.010761    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:58.162909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:58.162870    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:58.430724 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:58.430690    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:23:58.431113 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:58.430742    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:23:59.197182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:59.197147    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:59.430587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:59.430556    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:23:59.430748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:59.430609    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:23:59.550082 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:59.549999    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:23:59.694965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:23:59.694926    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:00.430319 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:00.430284    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:24:00.430514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:00.430354    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:24:01.430441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:01.430408    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-gdsfs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body=
Oct 29 23:24:01.430907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:01.430460    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs" podUID="ed0eab29-57b8-4012-8f45-04a0914fcd73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused"
Oct 29 23:24:02.951788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:02.951757    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:24:02.952243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:02.952229    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:24:02.953022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:02.953000    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-gdsfs"
Oct 29 23:24:23.144169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:23.144075    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:23.278852 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:23.278821    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:23.624125 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:23.624078    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:23.802212 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:23.802148    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:24.152487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:24.152456    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:24.324118 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:24.324080    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:24.502509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:24.502383    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:24.650157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:24.650116    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:24.885511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:24.885480    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:26.147305 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:24:26.147267    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:24:26.953195 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:26.953161    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:27.109742 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:27.109710    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:27.264395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:27.264322    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:27.900179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:27.900150    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:28.068116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:28.068066    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:28.225228 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:28.225161    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:28.377398 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:28.377370    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:28.922663 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:28.922634    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:29.105757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:29.105717    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:29.476818 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:29.476775    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:30.253579 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:30.253542    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:30.398179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:30.398147    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:31.438643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:31.438610    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:31.609995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:31.609783    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:31.961732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:31.961703    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:32.103740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:32.103712    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:32.525180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:32.525151    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:32.677695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:32.677645    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:33.510478 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:33.510446    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:33.666185 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:33.666144    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:33.817160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:33.817069    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:33.961505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:33.961458    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:35.330324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:35.330294    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:35.714445 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:35.714363    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:35.871971 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:35.871935    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:36.748996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:36.748946    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:36.916658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:36.916628    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:37.062647 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:37.062574    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:37.201404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:37.201377    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:38.656238 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:38.656211    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:38.799077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:38.799049    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:38.939183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:38.939108    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:39.844495 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:39.844467    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:40.020826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:40.020793    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:40.489048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:40.489010    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:40.651735 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:40.651702    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:40.823705 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:40.823674    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:42.072087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:42.072038    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:43.530187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:43.530154    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:43.677532 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:43.677502    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:45.564699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:45.564666    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:47.176551 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:47.176516    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:47.780252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:47.780220    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:47.933739 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:47.933713    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:48.095486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:48.095457    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:49.889153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.889119    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:24:49.892950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.892926    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.911300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.911276    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-032fa8\"/\"default-dockercfg-8rt48\""
Oct 29 23:24:49.925284 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.925257    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-032fa8\"/\"kube-root-ca.crt\""
Oct 29 23:24:49.925440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.925353    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-032fa8\"/\"openshift-service-ca.crt\""
Oct 29 23:24:49.926969 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.926947    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:24:49.958945 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.958917    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lknwv\" (UniqueName: \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.959100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.958955    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.959100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.959070    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.959100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.959091    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.959264 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.959120    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.959328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.959277    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:49.973410 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.973375    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-032fa8\"/\"simulator-w4-032fa8-kserve-self-signed-certs\""
Oct 29 23:24:49.986351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:49.986315    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:50.060612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060583    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.060767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060644    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.060767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060678    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lknwv\" (UniqueName: \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.060767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060707    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.060937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060764    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.060937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060802    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.061064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.060957    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.061127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.061094    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.061186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.061171    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.062956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.062933    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.063139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.063122    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.084419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.084396    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lknwv\" (UniqueName: \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\") pod \"simulator-w4-032fa8-kserve-55dfbf5775-k5dmz\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") " pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.168849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.168775    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:50.214741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.214713    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:50.515436 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:50.515410    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:24:50.517236 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:24:50.517200    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa1774dc_4bdb_443d_a0a6_be41b652a970.slice/crio-2e57d9e02f3072c858042af14cf322188de16c165ca081ea78d4c055b54b5f37 WatchSource:0}: Error finding container 2e57d9e02f3072c858042af14cf322188de16c165ca081ea78d4c055b54b5f37: Status 404 returned error can't find the container with id 2e57d9e02f3072c858042af14cf322188de16c165ca081ea78d4c055b54b5f37
Oct 29 23:24:51.352958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:51.352921    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" event={"ID":"aa1774dc-4bdb-443d-a0a6-be41b652a970","Type":"ContainerStarted","Data":"2e57d9e02f3072c858042af14cf322188de16c165ca081ea78d4c055b54b5f37"}
Oct 29 23:24:52.092953 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:52.092902    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:52.360618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:52.360534    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" event={"ID":"aa1774dc-4bdb-443d-a0a6-be41b652a970","Type":"ContainerStarted","Data":"b9d8529576d89463940ec066f9d0d3ac4c7bffb5c33e9cb142a82a78761f2b74"}
Oct 29 23:24:52.403812 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:52.403764    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podStartSLOduration=2.309146342 podStartE2EDuration="3.403740178s" totalImagesPullingTime="1.094593836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:24:52.402453883 +0000 UTC m=+703.488737018" watchObservedRunningTime="2026-10-29 23:24:52.403740178 +0000 UTC m=+703.490023311"
Oct 29 23:24:53.017158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:53.017127    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:53.171064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:53.171035    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:53.214941 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:53.214914    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:24:53.639795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:53.639762    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:53.807870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:53.807833    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:54.065171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:54.065099    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:55.731038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:55.731001    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:56.147606 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:24:56.147574    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:24:57.350011 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:57.349942    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:57.626343 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:57.626263    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:57.957586 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:57.957510    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:58.119940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:58.119894    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:58.454199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:58.454170    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:58.605202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:58.605165    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:59.140081 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.140040    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:59.370962 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.370931    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:24:59.374612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.374589    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.419249 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.419189    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-837245\"/\"kube-root-ca.crt\""
Oct 29 23:24:59.419993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.419956    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-837245\"/\"openshift-service-ca.crt\""
Oct 29 23:24:59.422726 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.422702    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:24:59.441739 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.441712    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.441852 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.441811    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.441903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.441849    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.442008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.441958    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.442102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.442080    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.442166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.442143    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltjz4\" (UniqueName: \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.482318 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.482284    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-837245\"/\"default-dockercfg-gx7pm\""
Oct 29 23:24:59.482318 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.482282    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-837245\"/\"simulator-w6-837245-kserve-self-signed-certs\""
Oct 29 23:24:59.506652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.506620    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:59.542815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.542788    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.542921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.542837    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.542921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.542892    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.542922    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.542990    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltjz4\" (UniqueName: \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.543045    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.543257    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543436 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.543410    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.543532 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.543342    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.545268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.545249    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.545432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.545416    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.572159 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.572137    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltjz4\" (UniqueName: \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\") pod \"simulator-w6-837245-kserve-6b7687ccbd-xx2s8\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") " pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.686651 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.686569    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:24:59.786899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.786868    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:59.945206 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.945076    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:24:59.992436 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:24:59.992409    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:24:59.994339 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:24:59.994313    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75f092d8_f18c_4cca_8c73_48ece7d668f9.slice/crio-d1ecda015ba51575fdbccd15ad51133fd1e2a5372b00a885e7d2e66d2327c11d WatchSource:0}: Error finding container d1ecda015ba51575fdbccd15ad51133fd1e2a5372b00a885e7d2e66d2327c11d: Status 404 returned error can't find the container with id d1ecda015ba51575fdbccd15ad51133fd1e2a5372b00a885e7d2e66d2327c11d
Oct 29 23:25:00.215878 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.215804    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:25:00.227950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.227927    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:25:00.400767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.400733    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" event={"ID":"75f092d8-f18c-4cca-8c73-48ece7d668f9","Type":"ContainerStarted","Data":"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"}
Oct 29 23:25:00.400914 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.400783    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" event={"ID":"75f092d8-f18c-4cca-8c73-48ece7d668f9","Type":"ContainerStarted","Data":"d1ecda015ba51575fdbccd15ad51133fd1e2a5372b00a885e7d2e66d2327c11d"}
Oct 29 23:25:00.412369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.412348    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:25:00.443157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.443120    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podStartSLOduration=1.443106947 podStartE2EDuration="1.443106947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:00.441869186 +0000 UTC m=+711.528152320" watchObservedRunningTime="2026-10-29 23:25:00.443106947 +0000 UTC m=+711.529390085"
Oct 29 23:25:00.687249 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:00.687219    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:25:01.167677 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.167642    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:25:01.173877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.173845    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.189926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.189900    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:25:01.202449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.202031    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"openshift-service-ca.crt\""
Oct 29 23:25:01.202710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.202686    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"kube-root-ca.crt\""
Oct 29 23:25:01.206000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.205966    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"simulator-w0-2670a5-kserve-self-signed-certs\""
Oct 29 23:25:01.206096 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.205994    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"default-dockercfg-dxlxd\""
Oct 29 23:25:01.261462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.261434    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:01.262098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262069    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.262214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262194    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.262275 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262254    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.262326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262283    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.262418 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262401    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.262477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.262440    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k679\" (UniqueName: \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.362998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.362955    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363014    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363042    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363067    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363090    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k679\" (UniqueName: \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363160    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363441    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363470    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.363540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.363487    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.365414 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.365385    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.365625 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.365609    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.384204 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.384175    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k679\" (UniqueName: \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\") pod \"simulator-w0-2670a5-kserve-6fc866d64-29tqg\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") " pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.486913 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.486853    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:01.784182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.784156    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:25:01.786697 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:01.786671    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf44ed094_369c_43e7_b834_1b3f071c4a65.slice/crio-64198c47e44b6c3831ded95e41469654c77f9107300cc9556908f6b098fb4ba3 WatchSource:0}: Error finding container 64198c47e44b6c3831ded95e41469654c77f9107300cc9556908f6b098fb4ba3: Status 404 returned error can't find the container with id 64198c47e44b6c3831ded95e41469654c77f9107300cc9556908f6b098fb4ba3
Oct 29 23:25:01.849020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:01.848953    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:02.413235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:02.413200    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" event={"ID":"f44ed094-369c-43e7-b834-1b3f071c4a65","Type":"ContainerStarted","Data":"aeac2d49973b890d42522fa7abf275773e0e08a789e37e587c3dbd272350c739"}
Oct 29 23:25:02.413235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:02.413236    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" event={"ID":"f44ed094-369c-43e7-b834-1b3f071c4a65","Type":"ContainerStarted","Data":"64198c47e44b6c3831ded95e41469654c77f9107300cc9556908f6b098fb4ba3"}
Oct 29 23:25:02.455605 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:02.455560    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podStartSLOduration=1.455546984 podStartE2EDuration="1.455546984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:02.454759398 +0000 UTC m=+713.541042531" watchObservedRunningTime="2026-10-29 23:25:02.455546984 +0000 UTC m=+713.541830117"
Oct 29 23:25:02.487386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:02.487362    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:03.348992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:03.348940    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:03.495158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:03.495127    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:04.056029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:04.055996    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:04.255471 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:04.255439    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:05.049344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.049311    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:25:05.053026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.052998    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.084375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.084347    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-be8726\"/\"simulator-w2-be8726-kserve-self-signed-certs\""
Oct 29 23:25:05.084592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.084374    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-be8726\"/\"openshift-service-ca.crt\""
Oct 29 23:25:05.084592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.084353    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-be8726\"/\"default-dockercfg-k8nz2\""
Oct 29 23:25:05.086955 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.086934    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:25:05.097901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.097882    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.098037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.097934    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.098037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.097966    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.098037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.098010    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd4fz\" (UniqueName: \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.098207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.098147    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.098207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.098185    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.168770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.168745    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:05.171218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.171201    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-be8726\"/\"kube-root-ca.crt\""
Oct 29 23:25:05.199764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.199733    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.199895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.199772    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd4fz\" (UniqueName: \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.199895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.199851    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.199894    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.200122    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.200217    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.200437    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.200546    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.200876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.200847    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.202585 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.202560    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.202826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.202808    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.232155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.232131    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd4fz\" (UniqueName: \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\") pod \"simulator-w2-be8726-kserve-6555f5746f-46lx9\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") " pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.368176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.368140    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:05.415882 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.415845    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:05.678682 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:05.678615    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:25:05.682723 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:05.682699    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c9c00e5_b490_4a4c_9d18_a73a2250fb72.slice/crio-5262ac56969a97bbcf1f46c4c75d1f362ddd94623607019f55f1960722c4ee05 WatchSource:0}: Error finding container 5262ac56969a97bbcf1f46c4c75d1f362ddd94623607019f55f1960722c4ee05: Status 404 returned error can't find the container with id 5262ac56969a97bbcf1f46c4c75d1f362ddd94623607019f55f1960722c4ee05
Oct 29 23:25:06.437207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:06.437169    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" event={"ID":"0c9c00e5-b490-4a4c-9d18-a73a2250fb72","Type":"ContainerStarted","Data":"d41dbca38ab29946fe8c4e88df7df14fa94773487a5244e5da4e8e254128774a"}
Oct 29 23:25:06.437207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:06.437211    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" event={"ID":"0c9c00e5-b490-4a4c-9d18-a73a2250fb72","Type":"ContainerStarted","Data":"5262ac56969a97bbcf1f46c4c75d1f362ddd94623607019f55f1960722c4ee05"}
Oct 29 23:25:06.481274 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:06.481227    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podStartSLOduration=1.481216021 podStartE2EDuration="1.481216021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:06.479580314 +0000 UTC m=+717.565863448" watchObservedRunningTime="2026-10-29 23:25:06.481216021 +0000 UTC m=+717.567499193"
Oct 29 23:25:07.020108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:07.020073    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:07.368842 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:07.368810    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:08.479542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:08.479513    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:09.687405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:09.687371    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:25:09.700367 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:09.700340    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:25:10.067657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.067586    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:10.467751 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.467708    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:25:10.570038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.570004    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:25:10.578820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.578795    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.597848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.597552    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0b2ae9\"/\"kube-root-ca.crt\""
Oct 29 23:25:10.598385 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.598353    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-0b2ae9\"/\"openshift-service-ca.crt\""
Oct 29 23:25:10.598491 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.598473    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0b2ae9\"/\"default-dockercfg-ccssv\""
Oct 29 23:25:10.598580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.598563    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0b2ae9\"/\"simulator-w5-0b2ae9-kserve-self-signed-certs\""
Oct 29 23:25:10.601720 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.599625    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:25:10.653667 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653619    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4z97\" (UniqueName: \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.653821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653690    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.653821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653722    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.653821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653754    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.653821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653779    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.653821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.653806    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.665872 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.665841    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:10.755327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755247    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755285    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755309    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755331    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755362    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755523    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4z97\" (UniqueName: \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755711    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755744    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.755909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.755831    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.757875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.757851    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.758218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.758201    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.776859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.776835    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4z97\" (UniqueName: \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\") pod \"simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") " pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:10.902914 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:10.902883    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:11.207939 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.207917    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:25:11.209277 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:11.209243    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3300e93_7cf1_407e_b46b_b43637553b86.slice/crio-5b3dbf09838592083c114150608e0d9fc03b28a5907b32156bfc4333e31b5628 WatchSource:0}: Error finding container 5b3dbf09838592083c114150608e0d9fc03b28a5907b32156bfc4333e31b5628: Status 404 returned error can't find the container with id 5b3dbf09838592083c114150608e0d9fc03b28a5907b32156bfc4333e31b5628
Oct 29 23:25:11.462697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.462613    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" event={"ID":"f3300e93-7cf1-407e-b46b-b43637553b86","Type":"ContainerStarted","Data":"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"}
Oct 29 23:25:11.462697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.462661    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" event={"ID":"f3300e93-7cf1-407e-b46b-b43637553b86","Type":"ContainerStarted","Data":"5b3dbf09838592083c114150608e0d9fc03b28a5907b32156bfc4333e31b5628"}
Oct 29 23:25:11.487779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.487750    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:11.500556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.500536    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:11.506604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.506568    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podStartSLOduration=1.506557763 podStartE2EDuration="1.506557763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:11.506075555 +0000 UTC m=+722.592358690" watchObservedRunningTime="2026-10-29 23:25:11.506557763 +0000 UTC m=+722.592840897"
Oct 29 23:25:11.596755 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.596730    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:11.904017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:11.903988    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:12.486234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:12.486205    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:25:12.587511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:12.587471    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:13.173903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:13.173873    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:14.678895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:14.678853    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:15.368808 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:15.368766    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:15.381280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:15.381258    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:15.496421 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:15.496399    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:25:16.103327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:16.103292    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:17.596312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:17.596269    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:18.840165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.840131    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:25:18.846001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.845954    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.861102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.861076    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:25:18.873332 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.873310    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9280e4\"/\"openshift-service-ca.crt\""
Oct 29 23:25:18.873332 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.873325    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-9280e4\"/\"kube-root-ca.crt\""
Oct 29 23:25:18.873475 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.873410    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9280e4\"/\"simulator-w3-9280e4-kserve-self-signed-certs\""
Oct 29 23:25:18.878245 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.878224    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-9280e4\"/\"default-dockercfg-gl7wv\""
Oct 29 23:25:18.924923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.924890    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.925061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.925040    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9l5\" (UniqueName: \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.925147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.925103    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.925147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.925121    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.925147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.925143    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.925302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.925215    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:18.927400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:18.927376    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:19.026420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026386    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9l5\" (UniqueName: \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026444    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026462    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026488    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026504    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026532    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026845    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.026947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.026919    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.027031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.027004    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.028820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.028786    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.029011 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.028994    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.049001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.048949    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9l5\" (UniqueName: \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\") pod \"simulator-w3-9280e4-kserve-c8fc9cd48-46s86\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") " pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.075874 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.075842    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:19.163317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.163256    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:19.509467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:19.508045    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:25:20.472205 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.472163    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:20.514660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.514629    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" event={"ID":"b4edd79f-1a7e-420b-a1b3-4ebd714f3352","Type":"ContainerStarted","Data":"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"}
Oct 29 23:25:20.514899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.514875    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" event={"ID":"b4edd79f-1a7e-420b-a1b3-4ebd714f3352","Type":"ContainerStarted","Data":"96b40ed3fee34276d698bcbd0690d90746ade5e7360bf2464a17f88e9c4db88a"}
Oct 29 23:25:20.559660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.559612    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podStartSLOduration=2.559599847 podStartE2EDuration="2.559599847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:20.558875205 +0000 UTC m=+731.645158340" watchObservedRunningTime="2026-10-29 23:25:20.559599847 +0000 UTC m=+731.645882980"
Oct 29 23:25:20.903570 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.903540    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:20.916438 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:20.916414    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:21.164307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:21.164232    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:21.531132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:21.531059    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:25:21.966246 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:21.966218    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:23.474266 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:23.474233    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:24.996574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:24.996526    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:26.147114 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:25:26.147079    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:25:26.571155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:26.571004    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:27.307131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:27.307103    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:28.063655 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:28.063626    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:29.163920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:29.163883    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:29.176449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:29.176426    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:29.508889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:29.508817    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:29.569192 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:29.569162    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:25:30.980764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:30.980731    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:32.468989 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:32.468942    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:33.928064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:33.928032    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:35.508612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.508575    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:25:35.513055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.513032    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.531643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.531617    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:25:35.543356 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.543329    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"premium-w0-2670a5-kserve-self-signed-certs\""
Oct 29 23:25:35.574931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.574903    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.575086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.574947    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.575086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.574987    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjfzh\" (UniqueName: \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.575240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.575098    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.575240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.575203    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.575240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.575236    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677304    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677369    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677395    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjfzh\" (UniqueName: \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677440    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677517    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.677736 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.677547    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.678272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.678243    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.678785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.678718    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.678927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.678880    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.680653 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.680619    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.683544 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.683520    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.700306 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.700282    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjfzh\" (UniqueName: \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\") pod \"premium-w0-2670a5-kserve-97dbfd669-5gtk9\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") " pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:35.825143 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:35.825114    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:36.127023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.126999    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:25:36.129012 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:36.128966    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfad574b_51c6_4b86_8a78_cf3d99faa747.slice/crio-a0e3d6b9089778e055d8e5a82ecfd7077df71d51e047097653da573e8ec6ba7c WatchSource:0}: Error finding container a0e3d6b9089778e055d8e5a82ecfd7077df71d51e047097653da573e8ec6ba7c: Status 404 returned error can't find the container with id a0e3d6b9089778e055d8e5a82ecfd7077df71d51e047097653da573e8ec6ba7c
Oct 29 23:25:36.446257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.446159    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:36.596554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.596519    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" event={"ID":"cfad574b-51c6-4b86-8a78-cf3d99faa747","Type":"ContainerStarted","Data":"b3c42c4741c2ab652d112a120ec5c552c53c4b369496e1ca5d542c7b1efad943"}
Oct 29 23:25:36.597000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.596562    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" event={"ID":"cfad574b-51c6-4b86-8a78-cf3d99faa747","Type":"ContainerStarted","Data":"a0e3d6b9089778e055d8e5a82ecfd7077df71d51e047097653da573e8ec6ba7c"}
Oct 29 23:25:36.621425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.621402    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:36.643381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.643338    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podStartSLOduration=1.643326412 podStartE2EDuration="1.643326412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:36.64273114 +0000 UTC m=+747.729014273" watchObservedRunningTime="2026-10-29 23:25:36.643326412 +0000 UTC m=+747.729609546"
Oct 29 23:25:36.825942 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.825912    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:36.899663 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:36.899632    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:38.558539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:38.558501    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:39.408711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.408681    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:25:39.412688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.412661    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.443568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.443537    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-be8726\"/\"premium-w2-be8726-kserve-self-signed-certs\""
Oct 29 23:25:39.445877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.445848    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:25:39.514600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.514566    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.514776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.514705    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.514776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.514751    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7szk\" (UniqueName: \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.514930 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.514908    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.515018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.514954    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.515018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.515004    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.518541 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.518507    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:39.615698 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615654    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615768    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615813    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7szk\" (UniqueName: \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615888    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615909    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.615931    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.616329    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.616374    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.616490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.616461    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.618165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.618142    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.618928 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.618905    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.637242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.637213    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7szk\" (UniqueName: \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\") pod \"premium-w2-be8726-kserve-7f9968956-ms448\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") " pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.729816 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.729745    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:39.964923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:39.964887    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:40.032530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:40.032508    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:25:40.034360 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:40.034328    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dfd907e_011e_46b5_a4f7_a53e8531a8be.slice/crio-3d5f3cfc66c2fa743a94e211bc226f19823f0d1142372e63333520ec3fa18de2 WatchSource:0}: Error finding container 3d5f3cfc66c2fa743a94e211bc226f19823f0d1142372e63333520ec3fa18de2: Status 404 returned error can't find the container with id 3d5f3cfc66c2fa743a94e211bc226f19823f0d1142372e63333520ec3fa18de2
Oct 29 23:25:40.620460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:40.620415    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" event={"ID":"2dfd907e-011e-46b5-a4f7-a53e8531a8be","Type":"ContainerStarted","Data":"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"}
Oct 29 23:25:40.620460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:40.620468    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" event={"ID":"2dfd907e-011e-46b5-a4f7-a53e8531a8be","Type":"ContainerStarted","Data":"3d5f3cfc66c2fa743a94e211bc226f19823f0d1142372e63333520ec3fa18de2"}
Oct 29 23:25:40.666227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:40.666180    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podStartSLOduration=1.666167412 podStartE2EDuration="1.666167412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:40.664447984 +0000 UTC m=+751.750731117" watchObservedRunningTime="2026-10-29 23:25:40.666167412 +0000 UTC m=+751.752450546"
Oct 29 23:25:40.730647 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:40.730621    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:41.592905 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:41.592872    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:43.252662 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:43.252622    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:43.399625 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:43.399595    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:45.090108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:45.090066    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:45.826220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:45.826185    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:45.839060 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:45.839031    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:46.058027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.057969    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:25:46.062506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.062481    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.079449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.079387    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-0b2ae9\"/\"premium-w5-0b2ae9-kserve-self-signed-certs\""
Oct 29 23:25:46.080146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.080129    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:25:46.144446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.144416    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:46.175580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.175551    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.175954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.175825    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.175954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.175906    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.176161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.175968    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpvb8\" (UniqueName: \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.176161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.176063    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.176161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.176094    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277258    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277315    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277397    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277436    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277481    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.277693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.277545    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpvb8\" (UniqueName: \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.278068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.278040    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.278312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.278196    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.278312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.278243    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.279875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.279847    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.280575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.280554    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.299164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.299142    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpvb8\" (UniqueName: \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\") pod \"premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") " pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.374983 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.374899    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:46.661125 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.661045    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:25:46.699170 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:46.699136    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:25:46.699496 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:46.699467    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0cd2e02_af0d_4ed4_922f_da7fac0a3726.slice/crio-2880f7f511a701bb6980d351e41b0c017473cd83afc9bce478142c2bc164fcd4 WatchSource:0}: Error finding container 2880f7f511a701bb6980d351e41b0c017473cd83afc9bce478142c2bc164fcd4: Status 404 returned error can't find the container with id 2880f7f511a701bb6980d351e41b0c017473cd83afc9bce478142c2bc164fcd4
Oct 29 23:25:47.128907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:47.128866    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:47.655429 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:47.655395    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" event={"ID":"a0cd2e02-af0d-4ed4-922f-da7fac0a3726","Type":"ContainerStarted","Data":"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"}
Oct 29 23:25:47.655901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:47.655442    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" event={"ID":"a0cd2e02-af0d-4ed4-922f-da7fac0a3726","Type":"ContainerStarted","Data":"2880f7f511a701bb6980d351e41b0c017473cd83afc9bce478142c2bc164fcd4"}
Oct 29 23:25:47.697345 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:47.697303    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podStartSLOduration=1.697290886 podStartE2EDuration="1.697290886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:47.696556065 +0000 UTC m=+758.782839199" watchObservedRunningTime="2026-10-29 23:25:47.697290886 +0000 UTC m=+758.783574017"
Oct 29 23:25:48.375948 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:48.375920    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:48.499638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:48.499604    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:49.730423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:49.730387    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:49.742697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:49.742671    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:50.173291 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:50.173257    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:50.682790 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:50.682762    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:25:51.558029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:51.558000    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:53.041922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:53.041871    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:53.206086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:53.206038    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:54.488540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:54.488506    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:55.921891 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:55.921858    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:56.148347 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:25:56.148314    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:25:56.375785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.375760    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:56.387957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.387934    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:56.713951 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.713873    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:25:56.717845 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.717822    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:25:56.718068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.718045    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.735153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.735131    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-313d9c\"/\"openshift-service-ca.crt\""
Oct 29 23:25:56.735153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.735143    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-313d9c\"/\"kube-root-ca.crt\""
Oct 29 23:25:56.748958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.748933    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-313d9c\"/\"default-dockercfg-5mnf7\""
Oct 29 23:25:56.749068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.748983    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-313d9c\"/\"premium-w1-313d9c-kserve-self-signed-certs\""
Oct 29 23:25:56.751132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.751115    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:25:56.828093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.828059    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:56.872392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872357    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.872554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872473    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwnh2\" (UniqueName: \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.872554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872516    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.872657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872633    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.872691 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872674    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.872822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.872806    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974034    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974108    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974134    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974229    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974269    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974330    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwnh2\" (UniqueName: \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974551    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974568    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.974589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.974551    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.976608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.976582    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.977145 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.977126    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:56.996268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:56.996245    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwnh2\" (UniqueName: \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\") pod \"premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") " pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:57.032256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.032230    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:57.332554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.332525    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:25:57.333935 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:25:57.333901    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4d66b98_2026_49a8_8f30_3fd0ee85b20c.slice/crio-bc6e35f87ce66a7bb88aa8d7538eb4a549c4f98ad268baa1b37756e30a4746d2 WatchSource:0}: Error finding container bc6e35f87ce66a7bb88aa8d7538eb4a549c4f98ad268baa1b37756e30a4746d2: Status 404 returned error can't find the container with id bc6e35f87ce66a7bb88aa8d7538eb4a549c4f98ad268baa1b37756e30a4746d2
Oct 29 23:25:57.389752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.389723    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:57.710770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.710686    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" event={"ID":"b4d66b98-2026-49a8-8f30-3fd0ee85b20c","Type":"ContainerStarted","Data":"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"}
Oct 29 23:25:57.710770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.710736    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" event={"ID":"b4d66b98-2026-49a8-8f30-3fd0ee85b20c","Type":"ContainerStarted","Data":"bc6e35f87ce66a7bb88aa8d7538eb4a549c4f98ad268baa1b37756e30a4746d2"}
Oct 29 23:25:57.759167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:57.759122    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podStartSLOduration=1.7591085560000002 podStartE2EDuration="1.759108556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:57.757281411 +0000 UTC m=+768.843564545" watchObservedRunningTime="2026-10-29 23:25:57.759108556 +0000 UTC m=+768.845391687"
Oct 29 23:25:58.032967 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:58.032899    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:25:59.265628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:59.265590    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:25:59.532263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:25:59.532176    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:00.894055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:00.894022    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:02.488538 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:02.488505    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:04.098948 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:04.098911    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:05.394661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:05.394631    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:06.725324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:06.725293    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:07.033348 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:07.033273    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:26:07.046168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:07.046142    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:26:07.771458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:07.771430    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:26:08.531920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:08.531890    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:10.168008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:10.167955    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:12.229635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:12.229591    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:13.815489 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:13.815455    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:15.815821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:15.815790    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:16.988771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:16.988736    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:18.415405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:18.415370    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:19.864497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.864465    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"]
Oct 29 23:26:19.869488 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.869462    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.885307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.885271    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.885432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.885317    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2pn6\" (UniqueName: \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.885580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.885555    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.895424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.895400    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"]
Oct 29 23:26:19.907184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.907161    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:26:19.907289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.907231    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:26:19.978541 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.978515    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:26:19.978668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.978625    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:26:19.986149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.986123    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.986241 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.986169    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.986357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.986344    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2pn6\" (UniqueName: \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.986865 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.986846    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:19.988574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:19.988553    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:20.007020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.006993    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2pn6\" (UniqueName: \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\") pod \"maas-api-revoke-sub-adf69316-7mr45\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:20.182800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.182738    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:20.289265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.289232    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:20.481031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.481006    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"]
Oct 29 23:26:20.482693 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:20.482661    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd0808ad_6aa3_4b8a_9370_590af9bff895.slice/crio-e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6 WatchSource:0}: Error finding container e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6: Status 404 returned error can't find the container with id e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6
Oct 29 23:26:20.829424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.829397    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45" event={"ID":"dd0808ad-6aa3-4b8a-9370-590af9bff895","Type":"ContainerStarted","Data":"e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6"}
Oct 29 23:26:20.932696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:20.932669    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:21.834067 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:21.834035    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:23.343025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:23.342995    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:23.848381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:23.848347    5111 generic.go:356] "Generic (PLEG): container finished" podID="dd0808ad-6aa3-4b8a-9370-590af9bff895" containerID="dfb777e597187c275b67aa975f164ae4e10af9961f6872934b262e0aa5843078" exitCode=0
Oct 29 23:26:23.848381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:23.848385    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45" event={"ID":"dd0808ad-6aa3-4b8a-9370-590af9bff895","Type":"ContainerDied","Data":"dfb777e597187c275b67aa975f164ae4e10af9961f6872934b262e0aa5843078"}
Oct 29 23:26:24.854190 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:24.854161    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:24.999363 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:24.999339    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:25.037248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.037223    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\") pod \"dd0808ad-6aa3-4b8a-9370-590af9bff895\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") "
Oct 29 23:26:25.037356 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.037307    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\") pod \"dd0808ad-6aa3-4b8a-9370-590af9bff895\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") "
Oct 29 23:26:25.037356 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.037337    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\" (UniqueName: \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\") pod \"dd0808ad-6aa3-4b8a-9370-590af9bff895\" (UID: \"dd0808ad-6aa3-4b8a-9370-590af9bff895\") "
Oct 29 23:26:25.037646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.037622    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca" pod "dd0808ad-6aa3-4b8a-9370-590af9bff895" (UID: "dd0808ad-6aa3-4b8a-9370-590af9bff895"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:26:25.039601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.039570    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6" pod "dd0808ad-6aa3-4b8a-9370-590af9bff895" (UID: "dd0808ad-6aa3-4b8a-9370-590af9bff895"). InnerVolumeSpecName "kube-api-access-r2pn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:26:25.039728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.039684    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca" pod "dd0808ad-6aa3-4b8a-9370-590af9bff895" (UID: "dd0808ad-6aa3-4b8a-9370-590af9bff895"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:26:25.138165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.138115    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:25.138165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.138138    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dd0808ad-6aa3-4b8a-9370-590af9bff895-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:25.138165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.138147    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2pn6\" (UniqueName: \"kubernetes.io/projected/dd0808ad-6aa3-4b8a-9370-590af9bff895-kube-api-access-r2pn6\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:25.859458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.859432    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"
Oct 29 23:26:25.859458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.859450    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45" event={"ID":"dd0808ad-6aa3-4b8a-9370-590af9bff895","Type":"ContainerDied","Data":"e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6"}
Oct 29 23:26:25.859918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:25.859514    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1194fd927a7ef97ac6a35ff0b13d4145acc1332033cca11e7f3c38a32b298f6"
Oct 29 23:26:26.149828 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:26:26.149747    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:26:26.227062 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:26.227029    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:27.884356 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:27.884316    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:29.466334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:29.466300    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:30.896315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:30.896286    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:32.065389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.065356    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"]
Oct 29 23:26:32.065925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.065907    5111 cpu_manager.go:414] "containerMap: removing container" podUID="dd0808ad-6aa3-4b8a-9370-590af9bff895" containerName="revoke-keys"
Oct 29 23:26:32.066114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.065928    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd0808ad-6aa3-4b8a-9370-590af9bff895" containerName="revoke-keys"
Oct 29 23:26:32.066114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.066110    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd0808ad-6aa3-4b8a-9370-590af9bff895" containerName="revoke-keys"
Oct 29 23:26:32.069401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.069379    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.089565 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.089541    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"]
Oct 29 23:26:32.100387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.100356    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:26:32.100387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.100357    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:26:32.100510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.100385    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:26:32.100510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.100373    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:26:32.103742 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.103717    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwtq\" (UniqueName: \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.103815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.103774    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.103884 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.103867    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.204826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.204789    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.204949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.204855    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.204949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.204906    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwtq\" (UniqueName: \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.205548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.205522    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.207099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.207078    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.226873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.226852    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwtq\" (UniqueName: \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\") pod \"maas-api-revoke-sub-a8a15ef4-kcrdx\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.381893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.381871    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:32.470280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.470250    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:32.682315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.682289    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"]
Oct 29 23:26:32.684362 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:32.684331    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3f91b6_1af9_4d67_9cbd_eb2a35dc44b7.slice/crio-f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf WatchSource:0}: Error finding container f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf: Status 404 returned error can't find the container with id f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf
Oct 29 23:26:32.894651 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:32.894621    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx" event={"ID":"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7","Type":"ContainerStarted","Data":"f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf"}
Oct 29 23:26:33.901304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:33.901273    5111 generic.go:356] "Generic (PLEG): container finished" podID="6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" containerID="2bc1e4d3c8af9c72d6a68918ee8098d3a79335631c532c7b751a547c6a5f5d61" exitCode=0
Oct 29 23:26:33.901671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:33.901327    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx" event={"ID":"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7","Type":"ContainerDied","Data":"2bc1e4d3c8af9c72d6a68918ee8098d3a79335631c532c7b751a547c6a5f5d61"}
Oct 29 23:26:33.916412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:33.916384    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:35.056043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.055999    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:35.131025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.131000    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\") pod \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") "
Oct 29 23:26:35.131182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.131086    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\" (UniqueName: \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\") pod \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") "
Oct 29 23:26:35.131182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.131113    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\") pod \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\" (UID: \"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7\") "
Oct 29 23:26:35.131522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.131497    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca" pod "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" (UID: "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:26:35.132993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.132949    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca" pod "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" (UID: "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:26:35.133057 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.133034    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq" pod "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" (UID: "6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7"). InnerVolumeSpecName "kube-api-access-cpwtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:26:35.232235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.232186    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:35.232235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.232206    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwtq\" (UniqueName: \"kubernetes.io/projected/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-kube-api-access-cpwtq\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:35.232235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.232216    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:35.608076 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.608046    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"]
Oct 29 23:26:35.608513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.608496    5111 cpu_manager.go:414] "containerMap: removing container" podUID="6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" containerName="revoke-keys"
Oct 29 23:26:35.608513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.608513    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" containerName="revoke-keys"
Oct 29 23:26:35.608656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.608626    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" containerName="revoke-keys"
Oct 29 23:26:35.611811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.611793    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.616931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.616905    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:35.629946 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.629925    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"]
Oct 29 23:26:35.736401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.736379    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl952\" (UniqueName: \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.736523 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.736435    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.736570 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.736543    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.837027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.836969    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.837124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.837076    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl952\" (UniqueName: \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.837183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.837154    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.837710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.837690    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.839500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.839483    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.861733 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.861674    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl952\" (UniqueName: \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\") pod \"maas-api-revoke-sub-f506b116-7xvzk\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:35.912266 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.912244    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx" event={"ID":"6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7","Type":"ContainerDied","Data":"f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf"}
Oct 29 23:26:35.912266 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.912263    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"
Oct 29 23:26:35.912444 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.912291    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9de661c80076bd990c97d63e77e3a696c3f731fd4a9ffa8bf1cf72395c61bbf"
Oct 29 23:26:35.925236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:35.925220    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:36.225374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:36.225352    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"]
Oct 29 23:26:36.227221 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:36.227190    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e4932e_c3a5_4d60_8194_9c7ea47fefb3.slice/crio-3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96 WatchSource:0}: Error finding container 3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96: Status 404 returned error can't find the container with id 3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96
Oct 29 23:26:36.903354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:36.903325    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:36.918472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:36.918442    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk" event={"ID":"66e4932e-c3a5-4d60-8194-9c7ea47fefb3","Type":"ContainerStarted","Data":"877297882d3ba39ceef0ab2f86f50b4e6b1e532d8fd2e538b8ca48aa4078078f"}
Oct 29 23:26:36.918601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:36.918498    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk" event={"ID":"66e4932e-c3a5-4d60-8194-9c7ea47fefb3","Type":"ContainerStarted","Data":"3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96"}
Oct 29 23:26:36.963853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:36.963800    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk" podStartSLOduration=1.9637829500000001 podStartE2EDuration="1.96378295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:36.962625213 +0000 UTC m=+808.048908347" watchObservedRunningTime="2026-10-29 23:26:36.96378295 +0000 UTC m=+808.050066085"
Oct 29 23:26:37.937018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:37.928335    5111 generic.go:356] "Generic (PLEG): container finished" podID="66e4932e-c3a5-4d60-8194-9c7ea47fefb3" containerID="877297882d3ba39ceef0ab2f86f50b4e6b1e532d8fd2e538b8ca48aa4078078f" exitCode=0
Oct 29 23:26:37.937018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:37.928514    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk" event={"ID":"66e4932e-c3a5-4d60-8194-9c7ea47fefb3","Type":"ContainerDied","Data":"877297882d3ba39ceef0ab2f86f50b4e6b1e532d8fd2e538b8ca48aa4078078f"}
Oct 29 23:26:38.940015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:38.936311    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:39.148353 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.148328    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:39.169815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.169790    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:26:39.170252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.170236    5111 cpu_manager.go:414] "containerMap: removing container" podUID="66e4932e-c3a5-4d60-8194-9c7ea47fefb3" containerName="revoke-keys"
Oct 29 23:26:39.170252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.170251    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66e4932e-c3a5-4d60-8194-9c7ea47fefb3" containerName="revoke-keys"
Oct 29 23:26:39.170411 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.170348    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="66e4932e-c3a5-4d60-8194-9c7ea47fefb3" containerName="revoke-keys"
Oct 29 23:26:39.175014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.174952    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.199812 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.199769    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:26:39.200074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.200051    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-032fa8-openshift-default-dockercfg-kpn7m\""
Oct 29 23:26:39.274744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.274715    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\") pod \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") "
Oct 29 23:26:39.274929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.274778    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\" (UniqueName: \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\") pod \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") "
Oct 29 23:26:39.274929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.274859    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\") pod \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\" (UID: \"66e4932e-c3a5-4d60-8194-9c7ea47fefb3\") "
Oct 29 23:26:39.275167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275138    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca" pod "66e4932e-c3a5-4d60-8194-9c7ea47fefb3" (UID: "66e4932e-c3a5-4d60-8194-9c7ea47fefb3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:26:39.275266 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275220    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275263    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275288    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxslj\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275347    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275411    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275586    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275628    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275664    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275799 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275692    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.275858 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.275798    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:39.277991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.277166    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952" pod "66e4932e-c3a5-4d60-8194-9c7ea47fefb3" (UID: "66e4932e-c3a5-4d60-8194-9c7ea47fefb3"). InnerVolumeSpecName "kube-api-access-kl952". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:26:39.278348 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.278326    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca" pod "66e4932e-c3a5-4d60-8194-9c7ea47fefb3" (UID: "66e4932e-c3a5-4d60-8194-9c7ea47fefb3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:26:39.376940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.376904    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.376940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.376943    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.376967    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxslj\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377050    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377429 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377410    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377436    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377439    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377520    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377552    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377589    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377618    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.377892 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377728    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl952\" (UniqueName: \"kubernetes.io/projected/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-kube-api-access-kl952\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:39.377892 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.377746    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/66e4932e-c3a5-4d60-8194-9c7ea47fefb3-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:39.378099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.378077    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.378404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.378380    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.378663 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.378627    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.380297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.380255    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.380431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.380413    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.398518 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.398492    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.398617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.398569    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxslj\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.517511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.517449    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:39.849295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.849265    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:26:39.850059 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:39.850028    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod016689f1_e9d0_40dd_a847_7607f990d01a.slice/crio-7198024a96ee62f48f3017ac1799f3beefe19be3b8189d72fe5d4d1c69ca346a WatchSource:0}: Error finding container 7198024a96ee62f48f3017ac1799f3beefe19be3b8189d72fe5d4d1c69ca346a: Status 404 returned error can't find the container with id 7198024a96ee62f48f3017ac1799f3beefe19be3b8189d72fe5d4d1c69ca346a
Oct 29 23:26:39.852470 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.852435    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:39.852586 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.852513    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:39.852652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.852588    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:39.944875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.944839    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" event={"ID":"016689f1-e9d0-40dd-a847-7607f990d01a","Type":"ContainerStarted","Data":"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"}
Oct 29 23:26:39.945341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.944902    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" event={"ID":"016689f1-e9d0-40dd-a847-7607f990d01a","Type":"ContainerStarted","Data":"7198024a96ee62f48f3017ac1799f3beefe19be3b8189d72fe5d4d1c69ca346a"}
Oct 29 23:26:39.947163 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.947135    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"
Oct 29 23:26:39.947281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.947164    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk" event={"ID":"66e4932e-c3a5-4d60-8194-9c7ea47fefb3","Type":"ContainerDied","Data":"3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96"}
Oct 29 23:26:39.947281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.947208    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fd633e95c26bc479441c82bb0465a55bfb0402c2c6bf62400fd368d9749be96"
Oct 29 23:26:39.995425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:39.995376    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podStartSLOduration=0.995362858 podStartE2EDuration="995.362858ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:39.99340055 +0000 UTC m=+811.079683675" watchObservedRunningTime="2026-10-29 23:26:39.995362858 +0000 UTC m=+811.081645991"
Oct 29 23:26:40.517617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:40.517586    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:40.519411 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:40.519383    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:40.519570 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:40.519436    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:40.616279 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:40.616247    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:41.518477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:41.518443    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:41.518872 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:41.518515    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:42.344770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:42.344735    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:42.517746 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:42.517714    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:42.517915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:42.517769    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:43.518446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:43.518412    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:43.518838 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:43.518464    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:44.144252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:44.144217    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:44.518208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:44.518128    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:44.518208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:44.518184    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:45.518728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.518694    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Oct 29 23:26:45.519170 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.518745    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Oct 29 23:26:45.571911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.571886    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"]
Oct 29 23:26:45.598883 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.598855    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.610395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.610366    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"]
Oct 29 23:26:45.620993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.620955    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:26:45.621117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.621047    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:26:45.621117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.621105    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:26:45.634466 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.634443    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:26:45.649269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.649247    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.649363 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.649302    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85nbl\" (UniqueName: \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.649405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.649377    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.750783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.750751    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.750954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.750810    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85nbl\" (UniqueName: \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.750954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.750839    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.751433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.751412    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.753233 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.753211    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.795780 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.795687    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85nbl\" (UniqueName: \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\") pod \"maas-api-revoke-sub-cfa936f8-hrf8n\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:45.901216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.901172    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:45.920277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:45.920232    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:46.241300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.241272    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"]
Oct 29 23:26:46.243431 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:46.243405    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6785f52_e79c_42ee_88c7_62c15035a994.slice/crio-2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86 WatchSource:0}: Error finding container 2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86: Status 404 returned error can't find the container with id 2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86
Oct 29 23:26:46.638655 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.638622    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:46.639908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.639882    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:46.640594 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.640572    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:26:46.990386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.990306    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n" event={"ID":"a6785f52-e79c-42ee-88c7-62c15035a994","Type":"ContainerStarted","Data":"a05e58fa55a3416882aeb6b741e4f1e53e405ee6a63175f210f3de885143b516"}
Oct 29 23:26:46.990386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:46.990371    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n" event={"ID":"a6785f52-e79c-42ee-88c7-62c15035a994","Type":"ContainerStarted","Data":"2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86"}
Oct 29 23:26:47.050403 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.050355    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n" podStartSLOduration=2.050342283 podStartE2EDuration="2.050342283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:47.047876492 +0000 UTC m=+818.134159626" watchObservedRunningTime="2026-10-29 23:26:47.050342283 +0000 UTC m=+818.136625417"
Oct 29 23:26:47.546209 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.546115    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:47.895161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.895126    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:26:47.898878 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.898859    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.921931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.921903    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:26:47.932074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.932054    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-313d9c\"/\"distinct-w1-313d9c-kserve-self-signed-certs\""
Oct 29 23:26:47.978585 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.978553    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.978745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.978613    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.978745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.978732    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.978842 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.978764    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.978889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.978838    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj9tb\" (UniqueName: \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.979026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.979001    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:47.998322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.998294    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:47.998918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.998889    5111 generic.go:356] "Generic (PLEG): container finished" podID="a6785f52-e79c-42ee-88c7-62c15035a994" containerID="a05e58fa55a3416882aeb6b741e4f1e53e405ee6a63175f210f3de885143b516" exitCode=0
Oct 29 23:26:47.999026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:47.998982    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n" event={"ID":"a6785f52-e79c-42ee-88c7-62c15035a994","Type":"ContainerDied","Data":"a05e58fa55a3416882aeb6b741e4f1e53e405ee6a63175f210f3de885143b516"}
Oct 29 23:26:48.079966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.079927    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080017    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080084    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj9tb\" (UniqueName: \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080153    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080252    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080299    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080369    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.080636 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.080613    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.081331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.081304    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.084004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.083952    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.086911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.086353    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.102645 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.102615    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj9tb\" (UniqueName: \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\") pod \"distinct-w1-313d9c-kserve-7ff4466df5-plrnp\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") " pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.214457 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.214382    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:48.516272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:48.516247    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:26:48.517047 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:48.517018    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode24f630f_ccc6_432b_8ff9_636d959f2980.slice/crio-43b30217faefbee23bf1629aae12b93ad5e9d96ed1919acdee19e895ca349253 WatchSource:0}: Error finding container 43b30217faefbee23bf1629aae12b93ad5e9d96ed1919acdee19e895ca349253: Status 404 returned error can't find the container with id 43b30217faefbee23bf1629aae12b93ad5e9d96ed1919acdee19e895ca349253
Oct 29 23:26:49.003414 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.003376    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:49.006026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.005968    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" event={"ID":"e24f630f-ccc6-432b-8ff9-636d959f2980","Type":"ContainerStarted","Data":"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"}
Oct 29 23:26:49.006150 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.006039    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" event={"ID":"e24f630f-ccc6-432b-8ff9-636d959f2980","Type":"ContainerStarted","Data":"43b30217faefbee23bf1629aae12b93ad5e9d96ed1919acdee19e895ca349253"}
Oct 29 23:26:49.052696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.052640    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podStartSLOduration=2.052623075 podStartE2EDuration="2.052623075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:49.049846905 +0000 UTC m=+820.136130039" watchObservedRunningTime="2026-10-29 23:26:49.052623075 +0000 UTC m=+820.138906210"
Oct 29 23:26:49.158276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.158249    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:49.215195 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.215161    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:49.293869 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.293791    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\") pod \"a6785f52-e79c-42ee-88c7-62c15035a994\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") "
Oct 29 23:26:49.294038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.293881    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\" (UniqueName: \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\") pod \"a6785f52-e79c-42ee-88c7-62c15035a994\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") "
Oct 29 23:26:49.294038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.294000    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\") pod \"a6785f52-e79c-42ee-88c7-62c15035a994\" (UID: \"a6785f52-e79c-42ee-88c7-62c15035a994\") "
Oct 29 23:26:49.294371 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.294337    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca" pod "a6785f52-e79c-42ee-88c7-62c15035a994" (UID: "a6785f52-e79c-42ee-88c7-62c15035a994"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:26:49.294487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.294463    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:49.295870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.295850    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca" pod "a6785f52-e79c-42ee-88c7-62c15035a994" (UID: "a6785f52-e79c-42ee-88c7-62c15035a994"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:26:49.296085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.296068    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl" pod "a6785f52-e79c-42ee-88c7-62c15035a994" (UID: "a6785f52-e79c-42ee-88c7-62c15035a994"). InnerVolumeSpecName "kube-api-access-85nbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:26:49.396089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.396057    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a6785f52-e79c-42ee-88c7-62c15035a994-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:49.396089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:49.396086    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85nbl\" (UniqueName: \"kubernetes.io/projected/a6785f52-e79c-42ee-88c7-62c15035a994-kube-api-access-85nbl\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:26:50.012778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:50.012748    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"
Oct 29 23:26:50.012778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:50.012772    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n" event={"ID":"a6785f52-e79c-42ee-88c7-62c15035a994","Type":"ContainerDied","Data":"2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86"}
Oct 29 23:26:50.013227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:50.012827    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c6f06a64f68a470bdcc964e3dc60d2d244ba17fbe73fbc0b5d09fdcba3a2a86"
Oct 29 23:26:50.923894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:50.923863    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:52.032522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:52.032487    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:54.084998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.083716    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:26:54.084998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.084454    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a6785f52-e79c-42ee-88c7-62c15035a994" containerName="revoke-keys"
Oct 29 23:26:54.084998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.084473    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6785f52-e79c-42ee-88c7-62c15035a994" containerName="revoke-keys"
Oct 29 23:26:54.084998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.084618    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6785f52-e79c-42ee-88c7-62c15035a994" containerName="revoke-keys"
Oct 29 23:26:54.093114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.093085    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.122192 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.122163    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:26:54.134144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.128592    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-be8726-openshift-default-dockercfg-mgctq\""
Oct 29 23:26:54.246137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246095    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246132    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246159    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246279    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246379    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246441    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246460    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc6v4\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246481    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.246604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.246530    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.348056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.347921    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.348321 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.348298    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.348441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.348412    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.348444    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc6v4\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349103    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349160    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349209    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349236    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349290    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349325    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.349932 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.349910    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.350311 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.350287    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.350454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.350323    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.350543 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.350354    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.352883 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.352858    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.357045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.357002    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.370586 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.370542    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc6v4\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.371181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.371163    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.445574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.445162    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:54.525045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.524997    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:26:54.534266 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.534239    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.539098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.539064    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:26:54.569997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.558013    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-313d9c-openshift-default-dockercfg-bgdwt\""
Oct 29 23:26:54.651014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.640520    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.660991    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661037    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661074    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661118    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661162    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661214    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661286    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661358    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffn2h\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.662200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.661417    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762401    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffn2h\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762452    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762486    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762506    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762529    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762560    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762588    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762631    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.762931 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.762676    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.763074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.763058    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.763574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.763552    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.763814 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.763797    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.764096 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.764079    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.764179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.764085    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.765985 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.765950    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.766253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.766235    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.810653 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.810568    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.810818 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.810698    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffn2h\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.910845 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.910755    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:54.941086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.941048    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:26:54.945946 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.945902    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:54.946056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.946028    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:54.946095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:54.946058    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:55.114900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.114840    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:26:55.140590 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.140557    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:26:55.140712 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.140686    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.144535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.144503    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" event={"ID":"631fe478-a305-4be7-93e5-7759979632e5","Type":"ContainerStarted","Data":"5e6c179efa5441973c2544ed96861be0bbe465a0a8862a00615b9a140de6550d"}
Oct 29 23:26:55.159179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.159158    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2670a5-openshift-default-dockercfg-6sq5q\""
Oct 29 23:26:55.261317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.261291    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:26:55.263327 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:55.263299    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb62997a5_7b30_4018_92d9_1d78d2430963.slice/crio-3651c47128c2f77d68c7326343287b2a3487d3bf6b1b96f09b92f0cdf68a98d9 WatchSource:0}: Error finding container 3651c47128c2f77d68c7326343287b2a3487d3bf6b1b96f09b92f0cdf68a98d9: Status 404 returned error can't find the container with id 3651c47128c2f77d68c7326343287b2a3487d3bf6b1b96f09b92f0cdf68a98d9
Oct 29 23:26:55.265900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.265871    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:55.265960 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.265933    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:55.266005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.265961    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:55.271713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.271685    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.271790 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.271748    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.271892 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.271866    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.271940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.271918    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.272066 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.272036    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.272176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.272134    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.272242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.272173    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.272242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.272204    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.272242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.272228    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv7b2\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380552    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380628    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380673    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380710    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380749    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380831    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380869    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380904    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.380929    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv7b2\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.385633    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.386328    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.386630    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.386877    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.387143    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.396996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.391370    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.415994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.409654    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.419394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.419370    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.431000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.422904    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv7b2\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:55.463631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:55.461357    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:56.061032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.041472    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:26:56.066521 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:26:56.066491    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dad75f8_a173_45a3_a7c9_cb49b639a64c.slice/crio-7a05d6b3a34ac2558b9e136c1f2fe36d1509707915c8017a6eebe9a6f603526f WatchSource:0}: Error finding container 7a05d6b3a34ac2558b9e136c1f2fe36d1509707915c8017a6eebe9a6f603526f: Status 404 returned error can't find the container with id 7a05d6b3a34ac2558b9e136c1f2fe36d1509707915c8017a6eebe9a6f603526f
Oct 29 23:26:56.070574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.070540    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:56.070811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.070790    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:56.070966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.070948    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:26:56.160312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.160224    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" event={"ID":"631fe478-a305-4be7-93e5-7759979632e5","Type":"ContainerStarted","Data":"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"}
Oct 29 23:26:56.163124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.163096    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" event={"ID":"b62997a5-7b30-4018-92d9-1d78d2430963","Type":"ContainerStarted","Data":"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"}
Oct 29 23:26:56.163242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.163139    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" event={"ID":"b62997a5-7b30-4018-92d9-1d78d2430963","Type":"ContainerStarted","Data":"3651c47128c2f77d68c7326343287b2a3487d3bf6b1b96f09b92f0cdf68a98d9"}
Oct 29 23:26:56.164944 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.164899    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" event={"ID":"4dad75f8-a173-45a3-a7c9-cb49b639a64c","Type":"ContainerStarted","Data":"7a05d6b3a34ac2558b9e136c1f2fe36d1509707915c8017a6eebe9a6f603526f"}
Oct 29 23:26:56.231836 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:26:56.231593    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:26:56.319102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.303830    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podStartSLOduration=2.303808559 podStartE2EDuration="2.303808559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:56.225007274 +0000 UTC m=+827.311290407" watchObservedRunningTime="2026-10-29 23:26:56.303808559 +0000 UTC m=+827.390091697"
Oct 29 23:26:56.328440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.320808    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podStartSLOduration=2.3207847839999998 podStartE2EDuration="2.320784784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:56.303317118 +0000 UTC m=+827.389600278" watchObservedRunningTime="2026-10-29 23:26:56.320784784 +0000 UTC m=+827.407067919"
Oct 29 23:26:56.445649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.445621    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:26:56.447519 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.447494    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:26:56.447686 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.447541    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:26:56.733106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.731998    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:56.920197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.920145    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:26:56.922278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.922250    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:26:56.922513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:56.922470    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:26:57.270083 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.270043    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" event={"ID":"4dad75f8-a173-45a3-a7c9-cb49b639a64c","Type":"ContainerStarted","Data":"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"}
Oct 29 23:26:57.450366 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.450330    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:26:57.450607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.450575    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:26:57.467028 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.466782    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:26:57.469898 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.469414    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:26:57.469898 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.469473    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:26:57.922458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.922423    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:26:57.922760 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:57.922718    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:26:58.233347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.226341    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:58.266017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.265960    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:58.345001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.344143    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podStartSLOduration=3.344121327 podStartE2EDuration="3.344121327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:57.426424894 +0000 UTC m=+828.512708029" watchObservedRunningTime="2026-10-29 23:26:58.344121327 +0000 UTC m=+829.430404464"
Oct 29 23:26:58.366295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.366266    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:26:58.450640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.450604    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:26:58.451740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.451696    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:26:58.464996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.461787    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:26:58.464996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.461840    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:26:58.914041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.909436    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:58.914041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.912344    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:26:58.914041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:58.912396    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:26:59.451001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.449709    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:26:59.451001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.449779    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:26:59.464397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.462206    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:26:59.464397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.462261    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:26:59.651008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.648103    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:26:59.918409 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.918375    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:26:59.918727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:26:59.918696    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:27:00.449999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.447335    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:27:00.449999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.447398    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:27:00.461778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.461748    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:27:00.462230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.461800    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:27:00.745512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.745421    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:00.923350 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.923315    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:27:00.923599 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:00.923572    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:27:01.446997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.446543    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Oct 29 23:27:01.446997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.446619    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Oct 29 23:27:01.462644 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.462618    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:27:01.463010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.462678    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:27:01.926021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.920105    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:27:01.926021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:01.920161    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:27:02.461880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.461846    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Oct 29 23:27:02.462081 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.461898    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Oct 29 23:27:02.468152 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.468118    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:02.608970 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.608937    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:27:02.610743 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.610402    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:27:02.613534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.613500    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:27:02.911964 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.911925    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Oct 29 23:27:02.912189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:02.912015    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Oct 29 23:27:03.280551 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.280460    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:03.528601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.528562    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:27:03.529090 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.528949    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:27:03.530833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.530773    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:27:03.915552 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.915525    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:27:03.915805 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.915787    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:27:03.916513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:03.916487    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:27:04.514806 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:04.514774    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:06.150654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:06.150623    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:07.804818 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:07.804778    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:09.509608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.509534    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:09.745899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.745863    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"]
Oct 29 23:27:09.751782 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.751749    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.795993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.791384    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"]
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817415    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817482    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a5477d0-d8dc-421f-80f6-40029d523361-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817541    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817571    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817598    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817732    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817766    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817803    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a5477d0-d8dc-421f-80f6-40029d523361-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.819994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.817837    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn2zc\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-kube-api-access-fn2zc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.918608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918572    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.918795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918626    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.918795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918660    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a5477d0-d8dc-421f-80f6-40029d523361-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.918795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918694    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn2zc\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-kube-api-access-fn2zc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.918795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918749    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918795    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a5477d0-d8dc-421f-80f6-40029d523361-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918844    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918885    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.918910    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.919338    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.919864 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.919836    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.920301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.920280    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.921477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.921007    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.921614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.921424    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a5477d0-d8dc-421f-80f6-40029d523361-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.924274 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.924253    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a5477d0-d8dc-421f-80f6-40029d523361-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.924408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.924303    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a5477d0-d8dc-421f-80f6-40029d523361-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.942172 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.942113    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:09.943004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:09.942962    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn2zc\" (UniqueName: \"kubernetes.io/projected/1a5477d0-d8dc-421f-80f6-40029d523361-kube-api-access-fn2zc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-59689\" (UID: \"1a5477d0-d8dc-421f-80f6-40029d523361\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:10.079230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.079198    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:10.397668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.397635    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"]
Oct 29 23:27:10.398464 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:10.398437    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a5477d0_d8dc_421f_80f6_40029d523361.slice/crio-634401a8a6120ac81da2fabd4adc83163aee6259f0e527fa81e76fdc269950ff WatchSource:0}: Error finding container 634401a8a6120ac81da2fabd4adc83163aee6259f0e527fa81e76fdc269950ff: Status 404 returned error can't find the container with id 634401a8a6120ac81da2fabd4adc83163aee6259f0e527fa81e76fdc269950ff
Oct 29 23:27:10.400863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.400827    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:10.400984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.400893    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:10.400984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.400924    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:10.416520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.416486    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" event={"ID":"1a5477d0-d8dc-421f-80f6-40029d523361","Type":"ContainerStarted","Data":"634401a8a6120ac81da2fabd4adc83163aee6259f0e527fa81e76fdc269950ff"}
Oct 29 23:27:10.525147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:10.525097    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:11.031750 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:11.031707    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:11.423314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:11.423278    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" event={"ID":"1a5477d0-d8dc-421f-80f6-40029d523361","Type":"ContainerStarted","Data":"3e22eb0916b8c6181d3fe46c8f9cabdb2d22eac0e9f33c598012c54dd588e0c1"}
Oct 29 23:27:12.089995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:12.081923    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:12.089995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:12.082063    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:12.089995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:12.082106    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:12.823527 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:12.823491    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:13.084359 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:13.084275    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:13.089914 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:13.089881    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:14.086932 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:14.086898    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:14.087540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:14.087509    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:14.399154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:14.399048    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:15.083314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:15.083279    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:15.083587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:15.083562    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:16.086475 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:16.086437    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:16.087675 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:16.087647    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:16.194000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:16.190893    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:17.080484 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:17.080445    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-59689 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Oct 29 23:27:17.080654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:17.080529    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podUID="1a5477d0-d8dc-421f-80f6-40029d523361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Oct 29 23:27:17.569691 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:17.569610    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:18.966391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:18.966347    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:18.967133 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:18.967111    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:18.968965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:18.968942    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689"
Oct 29 23:27:19.035559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:19.035511    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-59689" podStartSLOduration=10.035499989 podStartE2EDuration="10.035499989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:11.510296274 +0000 UTC m=+842.596579399" watchObservedRunningTime="2026-10-29 23:27:19.035499989 +0000 UTC m=+850.121783123"
Oct 29 23:27:19.325355 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:19.325278    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:19.819373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:19.819292    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:21.478085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:21.478050    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:22.898751 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:22.898718    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:23.123674 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.123641    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:27:23.128196 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.128174    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.147144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.147118    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:27:23.157472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.157411    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-032fa8\"/\"distinct-w4-032fa8-kserve-self-signed-certs\""
Oct 29 23:27:23.201852 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.201829    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.202027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.201857    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.202027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.201896    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.202168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.202148    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.202223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.202191    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.202223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.202212    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psq6g\" (UniqueName: \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.215400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.215374    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:23.313214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313048    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.313214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313138    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.313214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313172    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq6g\" (UniqueName: \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.313620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313246    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.313620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313283    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.313620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.313367    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.318701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.316285    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.318701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.316874    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.318701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.317021    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.324243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.324212    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.328997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.327600    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.337520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.337494    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psq6g\" (UniqueName: \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\") pod \"distinct-w4-032fa8-kserve-dfb8f989c-6q947\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") " pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.441203 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.441135    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:23.743761 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:23.743733    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:27:23.744502 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:23.744471    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde3882a9_9c6c_4f3d_9a81_fac7a96140e3.slice/crio-b6b0e8e10dae8546c716f2fcdb031d3d6ba955d0512b0ae2495cd7ba6046a900 WatchSource:0}: Error finding container b6b0e8e10dae8546c716f2fcdb031d3d6ba955d0512b0ae2495cd7ba6046a900: Status 404 returned error can't find the container with id b6b0e8e10dae8546c716f2fcdb031d3d6ba955d0512b0ae2495cd7ba6046a900
Oct 29 23:27:24.325423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.325389    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:27:24.329401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.329383    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.364294 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.364263    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:27:24.393072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.393035    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:24.423256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.423224    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:27:24.424504 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424470    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424512    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xbc\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424617    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424801 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424757    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424816    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424876    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424905    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.424961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.424927    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.425105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.425068    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.428101 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.428085    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.447375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.447339    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-9280e4-openshift-default-dockercfg-2jl2d\""
Oct 29 23:27:24.458108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.452162    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:27:24.513744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.513706    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" event={"ID":"de3882a9-9c6c-4f3d-9a81-fac7a96140e3","Type":"ContainerStarted","Data":"91b3dcd768420f1fd79c92a282a8b087bf5fdb9bba4c9f76bbefa0bfee55677c"}
Oct 29 23:27:24.513744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.513750    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" event={"ID":"de3882a9-9c6c-4f3d-9a81-fac7a96140e3","Type":"ContainerStarted","Data":"b6b0e8e10dae8546c716f2fcdb031d3d6ba955d0512b0ae2495cd7ba6046a900"}
Oct 29 23:27:24.525803 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.525773    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.525958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.525810    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.525958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.525840    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.525961    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526006    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xbc\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526026    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526062    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526097    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h629d\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526367 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526197    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526367 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526240    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526367 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526345    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526377    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526416    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526536    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526582    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526602    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526630    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526674    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526715    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.526927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526769    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.526927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.526772    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.527129 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.527110    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.527365 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.527342    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.528514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.528493    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.528679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.528659    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.549439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.549413    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.549631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.549613    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xbc\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.555575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.555537    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podStartSLOduration=1.5555271099999999 podStartE2EDuration="1.55552711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:24.555410409 +0000 UTC m=+855.641693544" watchObservedRunningTime="2026-10-29 23:27:24.55552711 +0000 UTC m=+855.641810244"
Oct 29 23:27:24.628078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628008    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628103    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628135    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628221    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628270    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h629d\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628356    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628386    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628515 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628447    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628515 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628504    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628548    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628507    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628854    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.628986 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.628952    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.629217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.629201    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.630175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.630158    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.630654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.630636    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.644567 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.644546    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:24.653924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.653893    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.654060 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.654042    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h629d\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.743352 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.743317    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:24.973808 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.973780    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:27:24.976027 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:24.975959    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f581c36_9808_47b8_9665_19b8d242ab62.slice/crio-1aa18bcd01504f2c21911b41a8aee93a576c076e169e516192f922cbb4734c72 WatchSource:0}: Error finding container 1aa18bcd01504f2c21911b41a8aee93a576c076e169e516192f922cbb4734c72: Status 404 returned error can't find the container with id 1aa18bcd01504f2c21911b41a8aee93a576c076e169e516192f922cbb4734c72
Oct 29 23:27:24.978742 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.978430    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:27:24.979465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.979429    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:24.979568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.979507    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:24.979568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.979547    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:24.983671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:24.983649    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.006646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.006620    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-837245-openshift-default-dockercfg-tn2ps\""
Oct 29 23:27:25.012166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.011262    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:27:25.033893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.033864    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.033991    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034052    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034120    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034197    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034233    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034320 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034270    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034320 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034300    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq6kk\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.034424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.034376    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.057124 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:25.057098    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00c6ed7b_e070_4cd0_bfd3_949afca12fa4.slice/crio-5217a9f1b741e49468cdb88cb9ee0cda50a4a25963d6a79f462e0fd22b9fff0f WatchSource:0}: Error finding container 5217a9f1b741e49468cdb88cb9ee0cda50a4a25963d6a79f462e0fd22b9fff0f: Status 404 returned error can't find the container with id 5217a9f1b741e49468cdb88cb9ee0cda50a4a25963d6a79f462e0fd22b9fff0f
Oct 29 23:27:25.063677 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.063648    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.063792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.063712    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.063792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.063756    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.065707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.065689    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:27:25.135287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135260    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135300    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135341    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135370    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135397    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135440    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq6kk\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135498    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135591    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135660    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135747    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.135795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.135763    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.136110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.136085    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.136478 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.136429    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.136823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.136799    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.138029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.138006    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.138460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.138446    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.156852 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.156828    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.157327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.157302    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq6kk\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.301528 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.301446    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:25.441235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.441203    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:25.523065 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.523027    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" event={"ID":"00c6ed7b-e070-4cd0-bfd3-949afca12fa4","Type":"ContainerStarted","Data":"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"}
Oct 29 23:27:25.523223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.523096    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" event={"ID":"00c6ed7b-e070-4cd0-bfd3-949afca12fa4","Type":"ContainerStarted","Data":"5217a9f1b741e49468cdb88cb9ee0cda50a4a25963d6a79f462e0fd22b9fff0f"}
Oct 29 23:27:25.527070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.527034    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" event={"ID":"0f581c36-9808-47b8-9665-19b8d242ab62","Type":"ContainerStarted","Data":"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"}
Oct 29 23:27:25.527070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.527075    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" event={"ID":"0f581c36-9808-47b8-9665-19b8d242ab62","Type":"ContainerStarted","Data":"1aa18bcd01504f2c21911b41a8aee93a576c076e169e516192f922cbb4734c72"}
Oct 29 23:27:25.594003 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.593731    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podStartSLOduration=1.593710596 podStartE2EDuration="1.593710596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:25.571252506 +0000 UTC m=+856.657535643" watchObservedRunningTime="2026-10-29 23:27:25.593710596 +0000 UTC m=+856.679993731"
Oct 29 23:27:25.624530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.624476    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podStartSLOduration=1.624460972 podStartE2EDuration="1.624460972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:25.623425123 +0000 UTC m=+856.709708257" watchObservedRunningTime="2026-10-29 23:27:25.624460972 +0000 UTC m=+856.710744108"
Oct 29 23:27:25.642047 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.639957    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:27:25.656048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.645061    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:25.660405 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:25.660359    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c56655_f6ef_4073_96ee_7836832a74ee.slice/crio-95beae5e5a2d12694be1ae0cbd5c4270be065b8a5de2457919c972fd03bffd5e WatchSource:0}: Error finding container 95beae5e5a2d12694be1ae0cbd5c4270be065b8a5de2457919c972fd03bffd5e: Status 404 returned error can't find the container with id 95beae5e5a2d12694be1ae0cbd5c4270be065b8a5de2457919c972fd03bffd5e
Oct 29 23:27:25.663622 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.663586    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.663709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.663669    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.663745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.663715    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:27:25.745165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:25.744966    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:26.014054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.012456    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:26.165338 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:27:26.165300    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:27:26.541091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.541049    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" event={"ID":"e1c56655-f6ef-4073-96ee-7836832a74ee","Type":"ContainerStarted","Data":"3790c0facfb272ead08611739289c5ed1ad0d08faf4ebcd99963f0fb13c060e5"}
Oct 29 23:27:26.543886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.541907    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" event={"ID":"e1c56655-f6ef-4073-96ee-7836832a74ee","Type":"ContainerStarted","Data":"95beae5e5a2d12694be1ae0cbd5c4270be065b8a5de2457919c972fd03bffd5e"}
Oct 29 23:27:26.625441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.624437    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podStartSLOduration=2.624418395 podStartE2EDuration="2.624418395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:26.618598195 +0000 UTC m=+857.704881357" watchObservedRunningTime="2026-10-29 23:27:26.624418395 +0000 UTC m=+857.710701531"
Oct 29 23:27:26.646725 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.646673    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:26.646859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.646748    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:26.745300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.745268    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:26.745453 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:26.745319    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:27.306007 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.303942    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:27.306007 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.304094    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:27.306007 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.304140    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:27.653402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.653360    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:27.653866 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.653435    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:27.748999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.744371    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:27.748999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:27.744435    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:28.308557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.308508    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:28.308833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.308789    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:28.455940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.455890    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:28.647999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.646366    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:28.647999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.646424    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:28.747998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.744357    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:28.747998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:28.744415    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:29.294997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.294441    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:29.304924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.304323    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:29.304924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.304383    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:29.647998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.646966    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:29.647998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.647045    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:29.746001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.745277    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:29.746001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:29.745335    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:30.314405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.314366    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:30.315019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.314963    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:30.648997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.645361    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:30.648997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.645424    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:30.749150 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.748663    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:30.749150 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.748727    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:30.934008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:30.933900    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:31.305482 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.305393    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:31.305731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.305700    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:31.645711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.645675    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Oct 29 23:27:31.646104 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.645739    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Oct 29 23:27:31.744075 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.744038    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Oct 29 23:27:31.744242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:31.744089    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Oct 29 23:27:32.304316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.304285    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Oct 29 23:27:32.304523 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.304344    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Oct 29 23:27:32.649246 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.649216    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:32.649639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.649621    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:32.650371 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.650348    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:27:32.738497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.738159    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:32.753909 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.753878    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:32.754337 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.754314    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:32.755243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:32.755224    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:27:33.434017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.431052    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:33.434017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.432126    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:33.442728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.442668    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:27:33.444797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.444034    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:33.465272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.465234    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:33.686034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:33.685936    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:27:35.006896 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:35.006862    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:36.757461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:36.757431    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:37.270021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:37.269960    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:38.215132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:38.215100    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:39.778461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:39.778420    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:41.388761 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:41.388727    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:43.038708 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:43.038668    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:44.806712 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:44.806678    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:46.562393 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:46.562348    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:48.429589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:48.429519    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:49.883834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:49.883727    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:51.372968 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:51.372935    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:53.187113 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:53.187058    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:54.566934 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.566898    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:27:54.571984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.571957    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.589610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.589583    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:27:54.599505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.599484    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2670a5\"/\"distinct-2-w0-2670a5-kserve-self-signed-certs\""
Oct 29 23:27:54.603578 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.603556    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:54.630798 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.630763    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.630937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.630822    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.630937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.630872    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.631084 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.631001    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.631084 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.631045    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfpx\" (UniqueName: \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.631252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.631233    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732550    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732651    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732703    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732728    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfpx\" (UniqueName: \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732787    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.732958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.732942    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.733427 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.733405    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.734284 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.734259    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.734787 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.734756    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.737375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.737348    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.765129 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.765093    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.766078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.766047    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfpx\" (UniqueName: \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\") pod \"distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") " pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:54.786714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.786676    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:54.885324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:54.885289    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:55.187154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:55.187128    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:27:55.187504 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:27:55.187480    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfcc2829d_357a_4385_9fcd_afdff00793eb.slice/crio-c6a6809ac5dbae75f6db67c63a09db1ff57674136d2183779a34f38b55ab60bf WatchSource:0}: Error finding container c6a6809ac5dbae75f6db67c63a09db1ff57674136d2183779a34f38b55ab60bf: Status 404 returned error can't find the container with id c6a6809ac5dbae75f6db67c63a09db1ff57674136d2183779a34f38b55ab60bf
Oct 29 23:27:55.799265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:55.799232    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" event={"ID":"fcc2829d-357a-4385-9fcd-afdff00793eb","Type":"ContainerStarted","Data":"b02794c01ca2a807d9e10dfb8146795e641433c3818942c92ff3d22173feb482"}
Oct 29 23:27:55.799265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:55.799271    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" event={"ID":"fcc2829d-357a-4385-9fcd-afdff00793eb","Type":"ContainerStarted","Data":"c6a6809ac5dbae75f6db67c63a09db1ff57674136d2183779a34f38b55ab60bf"}
Oct 29 23:27:55.841849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:55.841803    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podStartSLOduration=1.841790823 podStartE2EDuration="1.841790823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:55.841437689 +0000 UTC m=+886.927720823" watchObservedRunningTime="2026-10-29 23:27:55.841790823 +0000 UTC m=+886.928073988"
Oct 29 23:27:55.886328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:55.886306    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:27:56.148376 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:27:56.148342    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:27:56.267943 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:56.267916    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:57.742334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:57.742301    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:59.240177 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:59.240144    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:27:59.714360 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:27:59.714331    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:00.789017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:00.788968    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:02.135247 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:02.135217    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:02.809399 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:02.809361    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:03.682699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:03.682628    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:04.885753 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:04.885713    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:28:04.905821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:04.905791    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:28:05.235954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:05.235878    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:05.628088 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:05.628051    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:05.868741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:05.868715    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:28:06.413039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:06.413006    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:06.690520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:06.690429    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:08.278085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:08.278020    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:09.147587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.147559    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:28:09.151856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.151828    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:28:09.296422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.296396    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:28:09.301257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.301240    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.333925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.333897    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:28:09.401316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401237    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401275    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhplr\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401372    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401408    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401514    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401537    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401585    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401631    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.401733 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.401662    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.502870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.502842    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.502875    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhplr\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.502906    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503080    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503161    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503193    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503231    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503260    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503266    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503484 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503351    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503518    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503536    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503666 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503639    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.503986 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.503954    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.505390 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.505368    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.505580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.505564    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.524623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.524595    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.524744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.524669    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhplr\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.613854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.613828    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:09.642929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.642905    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:09.950946 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.950810    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:28:09.950946 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:09.950896    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee79c4c_aa1a_4bd8_968d_3e8df0501592.slice/crio-325eff6bf90010c11d20bd66063c7108e074367d9b85750ff5c13a49066e2689 WatchSource:0}: Error finding container 325eff6bf90010c11d20bd66063c7108e074367d9b85750ff5c13a49066e2689: Status 404 returned error can't find the container with id 325eff6bf90010c11d20bd66063c7108e074367d9b85750ff5c13a49066e2689
Oct 29 23:28:09.953226 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.953196    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:09.953300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.953256    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:09.953300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:09.953286    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:10.905037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:10.904995    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" event={"ID":"3ee79c4c-aa1a-4bd8-968d-3e8df0501592","Type":"ContainerStarted","Data":"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"}
Oct 29 23:28:10.905530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:10.905060    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" event={"ID":"3ee79c4c-aa1a-4bd8-968d-3e8df0501592","Type":"ContainerStarted","Data":"325eff6bf90010c11d20bd66063c7108e074367d9b85750ff5c13a49066e2689"}
Oct 29 23:28:10.966987 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:10.966925    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podStartSLOduration=1.966911568 podStartE2EDuration="1.966911568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:10.96481092 +0000 UTC m=+902.051094074" watchObservedRunningTime="2026-10-29 23:28:10.966911568 +0000 UTC m=+902.053194703"
Oct 29 23:28:11.190744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:11.190642    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:11.614129 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:11.614096    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:11.615606 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:11.615579    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:28:11.615745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:11.615627    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 29 23:28:12.527944 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:12.527911    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:12.614598 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:12.614562    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:28:12.614775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:12.614620    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 29 23:28:13.614234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:13.614195    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:28:13.614618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:13.614243    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 29 23:28:13.935685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:13.935608    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:14.614243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:14.614210    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:28:14.614722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:14.614261    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 29 23:28:15.565548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:15.565503    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:15.614949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:15.614916    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Oct 29 23:28:15.615355 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:15.615005    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Oct 29 23:28:16.962015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:16.961014    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:16.962015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:16.961953    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:16.963205 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:16.963180    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:28:17.087315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:17.087284    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:17.485768 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:17.485731    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:17.623038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:17.623004    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:18.446469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:18.446434    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:18.592209 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:18.592179    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:20.083070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:20.083039    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:21.605696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:21.605659    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:24.059170 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.059134    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:24.806961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.806925    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:28:24.811156 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.811136    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.839641 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.839619    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:28:24.876587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876536    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.876745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876716    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.876817 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876783    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.876864 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876810    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.876864 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876846    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.876952 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.876937    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.877039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.877023    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.877074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.877053    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psw7g\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.877116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.877085    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.978765 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.978728    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.978902 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.978787    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.978902 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.978817    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psw7g\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.978902 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.978848    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.978927    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979040    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979074    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979098    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979131    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979327 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979306    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979341    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979439    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979695    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.979853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.979835    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.981232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.981208    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:24.981341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:24.981323    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:25.000562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.000535    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:25.000689 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.000615    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psw7g\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:25.125310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.125282    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:25.444295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.444269    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:28:25.445357 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:25.445332    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe04ca7_5e9c_4738_9b70_59d000efe122.slice/crio-174c0b8bc1bb4f5a2ecad83d0b03c897d7314d3ed00a97f88c9e784649c026d8 WatchSource:0}: Error finding container 174c0b8bc1bb4f5a2ecad83d0b03c897d7314d3ed00a97f88c9e784649c026d8: Status 404 returned error can't find the container with id 174c0b8bc1bb4f5a2ecad83d0b03c897d7314d3ed00a97f88c9e784649c026d8
Oct 29 23:28:25.447786 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.447751    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:25.447863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.447817    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:25.447863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:25.447847    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:26.016534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.016494    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" event={"ID":"0fe04ca7-5e9c-4738-9b70-59d000efe122","Type":"ContainerStarted","Data":"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"}
Oct 29 23:28:26.016691 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.016557    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" event={"ID":"0fe04ca7-5e9c-4738-9b70-59d000efe122","Type":"ContainerStarted","Data":"174c0b8bc1bb4f5a2ecad83d0b03c897d7314d3ed00a97f88c9e784649c026d8"}
Oct 29 23:28:26.059967 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.059931    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:26.092008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.090541    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podStartSLOduration=2.090520471 podStartE2EDuration="2.090520471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:26.08949868 +0000 UTC m=+917.175781815" watchObservedRunningTime="2026-10-29 23:28:26.090520471 +0000 UTC m=+917.176803605"
Oct 29 23:28:26.125945 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.125903    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:26.127456 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.127423    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:26.127600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:26.127484    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:26.150089 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:28:26.150058    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:28:27.126367 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:27.126335    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:27.126744 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:27.126384    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:27.598351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:27.598319    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:28.126551 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:28.126514    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:28.126938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:28.126578    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:29.126215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:29.126185    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:29.126478 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:29.126236    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:29.333364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:29.333298    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:30.126431 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:30.126399    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:30.126661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:30.126640    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:30.960619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:30.960586    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:31.126584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:31.126555    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:31.126816 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:31.126794    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:32.125849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:32.125811    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Oct 29 23:28:32.126347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:32.125876    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Oct 29 23:28:32.685970 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:32.685927    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:33.129921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.129885    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:33.130415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.130268    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:33.130804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.130786    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:28:33.163060 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.163032    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"]
Oct 29 23:28:33.168236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.168217    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.185894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.177949    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"]
Oct 29 23:28:33.185894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.184249    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:28:33.185894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.184341    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:28:33.185894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.184371    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:28:33.185894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.184406    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:28:33.283188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.283151    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.283378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.283265    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.283378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.283335    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7jnv\" (UniqueName: \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.384445 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.384366    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.384589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.384453    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.384589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.384514    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7jnv\" (UniqueName: \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.385091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.385063    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.386708 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.386690    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.406737 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.406709    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7jnv\" (UniqueName: \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\") pod \"maas-api-revoke-sub-2114c313-r4ksf\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.495021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.494987    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:33.802440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:33.802417    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"]
Oct 29 23:28:33.805435 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:33.805404    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79512ae5_129d_474a_a1a6_cc575480c631.slice/crio-78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e WatchSource:0}: Error finding container 78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e: Status 404 returned error can't find the container with id 78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e
Oct 29 23:28:34.065543 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:34.065470    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf" event={"ID":"79512ae5-129d-474a-a1a6-cc575480c631","Type":"ContainerStarted","Data":"78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e"}
Oct 29 23:28:34.125047 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:34.125018    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:35.073099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:35.073068    5111 generic.go:356] "Generic (PLEG): container finished" podID="79512ae5-129d-474a-a1a6-cc575480c631" containerID="e0d550840b513cf21faa1c6c67a989cbe686b2c3c22334c9ea56801ccfdc7044" exitCode=0
Oct 29 23:28:35.073524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:35.073155    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf" event={"ID":"79512ae5-129d-474a-a1a6-cc575480c631","Type":"ContainerDied","Data":"e0d550840b513cf21faa1c6c67a989cbe686b2c3c22334c9ea56801ccfdc7044"}
Oct 29 23:28:35.749928 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:35.749897    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:36.221813 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.221789    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:36.322281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.322248    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\") pod \"79512ae5-129d-474a-a1a6-cc575480c631\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") "
Oct 29 23:28:36.322459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.322444    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\" (UniqueName: \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\") pod \"79512ae5-129d-474a-a1a6-cc575480c631\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") "
Oct 29 23:28:36.322555 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.322542    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\") pod \"79512ae5-129d-474a-a1a6-cc575480c631\" (UID: \"79512ae5-129d-474a-a1a6-cc575480c631\") "
Oct 29 23:28:36.322615 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.322556    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca" pod "79512ae5-129d-474a-a1a6-cc575480c631" (UID: "79512ae5-129d-474a-a1a6-cc575480c631"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:28:36.322896 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.322878    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:36.324526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.324498    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv" pod "79512ae5-129d-474a-a1a6-cc575480c631" (UID: "79512ae5-129d-474a-a1a6-cc575480c631"). InnerVolumeSpecName "kube-api-access-g7jnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:28:36.324620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.324531    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca" pod "79512ae5-129d-474a-a1a6-cc575480c631" (UID: "79512ae5-129d-474a-a1a6-cc575480c631"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:28:36.424440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.424418    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/79512ae5-129d-474a-a1a6-cc575480c631-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:36.424440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:36.424440    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7jnv\" (UniqueName: \"kubernetes.io/projected/79512ae5-129d-474a-a1a6-cc575480c631-kube-api-access-g7jnv\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:37.087349 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:37.087317    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf" event={"ID":"79512ae5-129d-474a-a1a6-cc575480c631","Type":"ContainerDied","Data":"78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e"}
Oct 29 23:28:37.087490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:37.087352    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"
Oct 29 23:28:37.087490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:37.087365    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="78cff36c6d803e1c74a9b9bc86f31aafd086774696ce424030de9e771d42b00e"
Oct 29 23:28:37.151722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:37.151687    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:38.518817 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:38.518759    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:39.891997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.890955    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:28:39.891997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.891704    5111 cpu_manager.go:414] "containerMap: removing container" podUID="79512ae5-129d-474a-a1a6-cc575480c631" containerName="revoke-keys"
Oct 29 23:28:39.891997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.891720    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79512ae5-129d-474a-a1a6-cc575480c631" containerName="revoke-keys"
Oct 29 23:28:39.891997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.891895    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="79512ae5-129d-474a-a1a6-cc575480c631" containerName="revoke-keys"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.921575    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.938201    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968615    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76898\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968661    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968692    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968716    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968805    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968837    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968866    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.968965    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:39.969835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:39.969083    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.089022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.081523    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.102005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.095662    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.212162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.212077    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.212922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.212897    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.214164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.214144    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.214772 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.214749    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76898\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.215462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.215441    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.215925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.215907    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.216459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.216440    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.252993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.214049    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253180 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.215856    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.095601    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253366 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.217258    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.102572    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.224619    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.253867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.253842    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.299996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.277522    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.307767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.307733    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76898\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\") pod \"e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") " pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:40.581442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:40.581177    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:41.807771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:41.807714    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:41.857160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:41.848350    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:28:41.891809 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:41.891769    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:41.892067 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:41.892036    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:41.892280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:41.892259    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:42.425379 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:42.425289    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" event={"ID":"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5","Type":"ContainerStarted","Data":"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"}
Oct 29 23:28:42.431891 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:42.431856    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" event={"ID":"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5","Type":"ContainerStarted","Data":"5aac787b18da78b6afa399615e63675767832d1424a8ba5e00e9b5bd2f76fbf1"}
Oct 29 23:28:42.504635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:42.504575    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podStartSLOduration=3.504556433 podStartE2EDuration="3.504556433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:42.486507412 +0000 UTC m=+933.572790547" watchObservedRunningTime="2026-10-29 23:28:42.504556433 +0000 UTC m=+933.590839568"
Oct 29 23:28:42.583775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:42.583745    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:43.602299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:43.602264    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:43.602729 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:43.602318    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:44.140090 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:44.140054    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:44.591346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:44.591312    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:44.591591 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:44.591563    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:45.597000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:45.590280    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:45.597000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:45.590337    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:45.753932 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:45.753899    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:46.164656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.164626    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"]
Oct 29 23:28:46.169054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.169030    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.203502 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.203471    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:28:46.204378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.204143    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:28:46.204378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.204253    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:28:46.205536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.205505    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"]
Oct 29 23:28:46.217056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.217036    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:28:46.268520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.268498    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8929\" (UniqueName: \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.268706 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.268687    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.268841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.268825    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.370100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.370065    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.370296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.370114    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8929\" (UniqueName: \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.370370 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.370329    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.370871 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.370837    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.394996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.390640    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.398640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.398604    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8929\" (UniqueName: \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\") pod \"maas-api-revoke-sub-af510c64-c9mlb\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.512678 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.512599    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:46.598997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.598566    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:46.598997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.598625    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:46.824819 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:46.824781    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"]
Oct 29 23:28:46.825245 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:46.825219    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda367053e_5414_4b97_8e0b_cf22e835ab14.slice/crio-4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2 WatchSource:0}: Error finding container 4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2: Status 404 returned error can't find the container with id 4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2
Oct 29 23:28:47.298930 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.298894    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:47.458841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.458756    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb" event={"ID":"a367053e-5414-4b97-8e0b-cf22e835ab14","Type":"ContainerStarted","Data":"5035e621cf52c62e96d479bbbf432cc04076e7d82e1252bf1a5ed44e7cfe9351"}
Oct 29 23:28:47.458841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.458818    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb" event={"ID":"a367053e-5414-4b97-8e0b-cf22e835ab14","Type":"ContainerStarted","Data":"4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2"}
Oct 29 23:28:47.510173 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.510124    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb" podStartSLOduration=1.510110751 podStartE2EDuration="1.510110751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:47.509730755 +0000 UTC m=+938.596013888" watchObservedRunningTime="2026-10-29 23:28:47.510110751 +0000 UTC m=+938.596393884"
Oct 29 23:28:47.597392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.597354    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:47.597589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:47.597432    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:48.467712 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:48.467677    5111 generic.go:356] "Generic (PLEG): container finished" podID="a367053e-5414-4b97-8e0b-cf22e835ab14" containerID="5035e621cf52c62e96d479bbbf432cc04076e7d82e1252bf1a5ed44e7cfe9351" exitCode=0
Oct 29 23:28:48.468200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:48.467755    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb" event={"ID":"a367053e-5414-4b97-8e0b-cf22e835ab14","Type":"ContainerDied","Data":"5035e621cf52c62e96d479bbbf432cc04076e7d82e1252bf1a5ed44e7cfe9351"}
Oct 29 23:28:48.584165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:48.584134    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Oct 29 23:28:48.584332 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:48.584186    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Oct 29 23:28:48.685394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:48.685359    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:49.643064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.643036    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:49.708714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.708685    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:49.710326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.710305    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:49.711166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.711142    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:28:49.738548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.738486    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\" (UniqueName: \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\") pod \"a367053e-5414-4b97-8e0b-cf22e835ab14\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") "
Oct 29 23:28:49.738548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.738524    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\") pod \"a367053e-5414-4b97-8e0b-cf22e835ab14\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") "
Oct 29 23:28:49.738693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.738636    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\") pod \"a367053e-5414-4b97-8e0b-cf22e835ab14\" (UID: \"a367053e-5414-4b97-8e0b-cf22e835ab14\") "
Oct 29 23:28:49.739200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.739176    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca" pod "a367053e-5414-4b97-8e0b-cf22e835ab14" (UID: "a367053e-5414-4b97-8e0b-cf22e835ab14"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:28:49.740789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.740763    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929" pod "a367053e-5414-4b97-8e0b-cf22e835ab14" (UID: "a367053e-5414-4b97-8e0b-cf22e835ab14"). InnerVolumeSpecName "kube-api-access-b8929". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:28:49.741044 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.741022    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca" pod "a367053e-5414-4b97-8e0b-cf22e835ab14" (UID: "a367053e-5414-4b97-8e0b-cf22e835ab14"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:28:49.840267 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.840231    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:49.840267 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.840266    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/a367053e-5414-4b97-8e0b-cf22e835ab14-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:49.840448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:49.840282    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8929\" (UniqueName: \"kubernetes.io/projected/a367053e-5414-4b97-8e0b-cf22e835ab14-kube-api-access-b8929\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:50.248547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.248513    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:50.481435 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.481365    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"
Oct 29 23:28:50.481435 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.481398    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb" event={"ID":"a367053e-5414-4b97-8e0b-cf22e835ab14","Type":"ContainerDied","Data":"4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2"}
Oct 29 23:28:50.481648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.481443    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ca2fb1f456b9defbf85d5f5043bfa5a14441a99241221086114f2fd7cbe4bf2"
Oct 29 23:28:50.963268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.963234    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"]
Oct 29 23:28:50.963775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.963761    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a367053e-5414-4b97-8e0b-cf22e835ab14" containerName="revoke-keys"
Oct 29 23:28:50.963775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.963775    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a367053e-5414-4b97-8e0b-cf22e835ab14" containerName="revoke-keys"
Oct 29 23:28:50.963881 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.963872    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a367053e-5414-4b97-8e0b-cf22e835ab14" containerName="revoke-keys"
Oct 29 23:28:50.967201 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.967183    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:50.984328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.984306    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:28:50.984464 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.984371    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:28:50.997993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.997955    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:28:50.998175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:50.998060    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:28:51.002960 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.002923    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"]
Oct 29 23:28:51.055245 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.055221    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rp9d\" (UniqueName: \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.055364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.055348    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.055417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.055371    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.156629 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.156598    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.156733 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.156640    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.157033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.157011    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rp9d\" (UniqueName: \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.157308 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.157286    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.159111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.159085    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.178695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.178670    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rp9d\" (UniqueName: \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\") pod \"maas-api-revoke-sub-44903331-vn4x2\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.280553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.280500    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:51.587429 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.587401    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"]
Oct 29 23:28:51.587736 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:51.587709    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b88976_0b31_49c3_865e_4ef71d883307.slice/crio-9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406 WatchSource:0}: Error finding container 9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406: Status 404 returned error can't find the container with id 9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406
Oct 29 23:28:51.732700 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:51.732665    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:52.495243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:52.495208    5111 generic.go:356] "Generic (PLEG): container finished" podID="28b88976-0b31-49c3-865e-4ef71d883307" containerID="1f424d2675e7ef18755d5f10ae38c660be6772d05f2927d5488666b99c705732" exitCode=0
Oct 29 23:28:52.495605 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:52.495255    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2" event={"ID":"28b88976-0b31-49c3-865e-4ef71d883307","Type":"ContainerDied","Data":"1f424d2675e7ef18755d5f10ae38c660be6772d05f2927d5488666b99c705732"}
Oct 29 23:28:52.495605 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:52.495311    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2" event={"ID":"28b88976-0b31-49c3-865e-4ef71d883307","Type":"ContainerStarted","Data":"9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406"}
Oct 29 23:28:53.280101 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.280056    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:53.656133 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.656110    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:53.789300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.789270    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\") pod \"28b88976-0b31-49c3-865e-4ef71d883307\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") "
Oct 29 23:28:53.789481 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.789349    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\") pod \"28b88976-0b31-49c3-865e-4ef71d883307\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") "
Oct 29 23:28:53.789481 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.789384    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\" (UniqueName: \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\") pod \"28b88976-0b31-49c3-865e-4ef71d883307\" (UID: \"28b88976-0b31-49c3-865e-4ef71d883307\") "
Oct 29 23:28:53.789737 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.789712    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca" pod "28b88976-0b31-49c3-865e-4ef71d883307" (UID: "28b88976-0b31-49c3-865e-4ef71d883307"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:28:53.791417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.791397    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d" pod "28b88976-0b31-49c3-865e-4ef71d883307" (UID: "28b88976-0b31-49c3-865e-4ef71d883307"). InnerVolumeSpecName "kube-api-access-6rp9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:28:53.791490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.791396    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca" pod "28b88976-0b31-49c3-865e-4ef71d883307" (UID: "28b88976-0b31-49c3-865e-4ef71d883307"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:28:53.890481 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.890460    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:53.890565 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.890481    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/28b88976-0b31-49c3-865e-4ef71d883307-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:53.890565 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:53.890497    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rp9d\" (UniqueName: \"kubernetes.io/projected/28b88976-0b31-49c3-865e-4ef71d883307-kube-api-access-6rp9d\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:28:54.445805 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.445775    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:28:54.446300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.446286    5111 cpu_manager.go:414] "containerMap: removing container" podUID="28b88976-0b31-49c3-865e-4ef71d883307" containerName="revoke-keys"
Oct 29 23:28:54.446357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.446301    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28b88976-0b31-49c3-865e-4ef71d883307" containerName="revoke-keys"
Oct 29 23:28:54.446439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.446430    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="28b88976-0b31-49c3-865e-4ef71d883307" containerName="revoke-keys"
Oct 29 23:28:54.449831 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.449816    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.490322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.490291    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:28:54.508588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.508558    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2" event={"ID":"28b88976-0b31-49c3-865e-4ef71d883307","Type":"ContainerDied","Data":"9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406"}
Oct 29 23:28:54.508588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.508582    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"
Oct 29 23:28:54.508758 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.508600    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9a8106fa65f4f9a519b2dfa57e7fe483477fa4c86af27b4dcbd919b3e0786406"
Oct 29 23:28:54.593270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.593242    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:28:54.597137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.597115    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.599867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.599837    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600007    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600101 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600043    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600162    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdlhl\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600239 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600200    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600254    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600357    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600424    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.600508 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.600456    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.622792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.622764    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:28:54.641843 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.641808    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:54.702341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702260    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.702341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702303    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702357    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702491    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702520    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702592    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702617    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.702792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702645    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702830    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdlhl\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702851    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702862    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702919    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702947    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97qkz\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.702993    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703018    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703041    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703162    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703225    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.703632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703251    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.703632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703279    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.703632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703292    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703426    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.703853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.703734    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.705344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.705316    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.705466 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.705437    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.726295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.726269    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.726647 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.726617    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdlhl\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\") pod \"e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") " pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.765647 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.765618    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:54.804487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804367    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.804487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804404    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.804487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804428    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.804487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804465    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804498    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804547    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804587    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804612    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804673    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97qkz\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.804822    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.805021    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.805058    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.805100    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.805642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.805520    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.807109 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.807085    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.807221 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.807203    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.834779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.834750    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.838368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.838342    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97qkz\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\") pod \"e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") " pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:54.913105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:54.913080    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:55.083824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.083798    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:28:55.084749 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:55.084719    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8aeec2e_d846_4de9_a12b_f673207514e4.slice/crio-e861d85b18b61a4e47d9066ed5c4b2343f6547eadb55803e1ea4b054a07be06c WatchSource:0}: Error finding container e861d85b18b61a4e47d9066ed5c4b2343f6547eadb55803e1ea4b054a07be06c: Status 404 returned error can't find the container with id e861d85b18b61a4e47d9066ed5c4b2343f6547eadb55803e1ea4b054a07be06c
Oct 29 23:28:55.087035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.087007    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.087098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.087067    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.087137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.087101    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.220185 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.219906    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:28:55.223323 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:55.223291    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4034b6_6cab_4dbd_85bd_3d4a53749064.slice/crio-8d56f95b191a30431f6a0f0b90a1fd3e19b1db2a39d01f372c0c87db7320eb9e WatchSource:0}: Error finding container 8d56f95b191a30431f6a0f0b90a1fd3e19b1db2a39d01f372c0c87db7320eb9e: Status 404 returned error can't find the container with id 8d56f95b191a30431f6a0f0b90a1fd3e19b1db2a39d01f372c0c87db7320eb9e
Oct 29 23:28:55.226217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.226184    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.226326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.226263    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.226326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.226307    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:28:55.516894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.516807    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" event={"ID":"0a4034b6-6cab-4dbd-85bd-3d4a53749064","Type":"ContainerStarted","Data":"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"}
Oct 29 23:28:55.516894 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.516869    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" event={"ID":"0a4034b6-6cab-4dbd-85bd-3d4a53749064","Type":"ContainerStarted","Data":"8d56f95b191a30431f6a0f0b90a1fd3e19b1db2a39d01f372c0c87db7320eb9e"}
Oct 29 23:28:55.518649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.518620    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" event={"ID":"a8aeec2e-d846-4de9-a12b-f673207514e4","Type":"ContainerStarted","Data":"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"}
Oct 29 23:28:55.518739 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.518670    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" event={"ID":"a8aeec2e-d846-4de9-a12b-f673207514e4","Type":"ContainerStarted","Data":"e861d85b18b61a4e47d9066ed5c4b2343f6547eadb55803e1ea4b054a07be06c"}
Oct 29 23:28:55.564702 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.564637    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podStartSLOduration=1.5646175260000001 podStartE2EDuration="1.564617526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:55.56107246 +0000 UTC m=+946.647355597" watchObservedRunningTime="2026-10-29 23:28:55.564617526 +0000 UTC m=+946.650900661"
Oct 29 23:28:55.610601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.610544    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podStartSLOduration=1.61053083 podStartE2EDuration="1.61053083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:55.607411024 +0000 UTC m=+946.693694159" watchObservedRunningTime="2026-10-29 23:28:55.61053083 +0000 UTC m=+946.696813963"
Oct 29 23:28:55.766445 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.766411    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:28:55.916000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:55.915076    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:28:56.150234 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:28:56.150201    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:28:56.158785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.158761    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:56.636240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.636069    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"]
Oct 29 23:28:56.649463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.649262    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.659438 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.659411    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"]
Oct 29 23:28:56.730077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.729780    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:28:56.730324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.730298    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:28:56.743164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.743138    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:28:56.743384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.743369    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:28:56.743486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.743453    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.743670 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.743652    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b42cb\" (UniqueName: \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.743845 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.743830    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.768016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.767988    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:28:56.768380 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.768038    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:28:56.845083 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.845045    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.845248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.845126    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b42cb\" (UniqueName: \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.845248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.845189    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.845907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.845887    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.849041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.847502    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.868467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.865462    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b42cb\" (UniqueName: \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\") pod \"maas-api-revoke-sub-75099af1-hdtfr\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:56.925287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.925179    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:28:56.925287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.925254    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:28:56.965028 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:56.965004    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:57.335432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.335404    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"]
Oct 29 23:28:57.336290 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:28:57.336266    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod624b2813_eb9a_47e6_8ba5_e8f00d2c35f7.slice/crio-8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476 WatchSource:0}: Error finding container 8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476: Status 404 returned error can't find the container with id 8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476
Oct 29 23:28:57.540746 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.540709    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr" event={"ID":"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7","Type":"ContainerStarted","Data":"8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476"}
Oct 29 23:28:57.766486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.766460    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:28:57.766616 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.766511    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:28:57.916998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.916335    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:28:57.916998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:57.916395    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:28:58.202995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.202024    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:58.548692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.548527    5111 generic.go:356] "Generic (PLEG): container finished" podID="624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" containerID="e9a19d04153865ec803168a30086850754d8a70e5ce35cabeb3211cd605228d7" exitCode=0
Oct 29 23:28:58.548692 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.548569    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr" event={"ID":"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7","Type":"ContainerDied","Data":"e9a19d04153865ec803168a30086850754d8a70e5ce35cabeb3211cd605228d7"}
Oct 29 23:28:58.766265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.766216    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:28:58.766441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.766289    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:28:58.922124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.922087    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:28:58.922658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:58.922156    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:28:59.507000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.506469    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:28:59.766385 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.766166    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:28:59.766385 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.766261    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:28:59.870082 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.870055    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:28:59.921563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.921531    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:28:59.921854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.921808    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:28:59.924765 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:28:59.924714    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:00.044709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.044607    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\") pod \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") "
Oct 29 23:29:00.044709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.044669    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\") pod \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") "
Oct 29 23:29:00.044936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.044779    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\" (UniqueName: \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\") pod \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\" (UID: \"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7\") "
Oct 29 23:29:00.045399 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.045368    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca" pod "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" (UID: "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:00.045650 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.045634    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:00.047184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.047158    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca" pod "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" (UID: "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:00.047405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.047381    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb" pod "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" (UID: "624b2813-eb9a-47e6-8ba5-e8f00d2c35f7"). InnerVolumeSpecName "kube-api-access-b42cb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:00.147261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.147232    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b42cb\" (UniqueName: \"kubernetes.io/projected/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-kube-api-access-b42cb\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:00.147455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.147439    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:00.223176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.223137    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:00.568187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.568143    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr" event={"ID":"624b2813-eb9a-47e6-8ba5-e8f00d2c35f7","Type":"ContainerDied","Data":"8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476"}
Oct 29 23:29:00.568454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.568252    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e155f561a26e9c5dc8097b7c1e00ecf1050999885c3ad5c38f1bf8052d0a476"
Oct 29 23:29:00.568454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.568254    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"
Oct 29 23:29:00.766413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.766253    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:29:00.766413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.766309    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:29:00.915325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.915289    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:29:00.915589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:00.915562    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:29:01.765924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:01.765889    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Oct 29 23:29:01.766391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:01.765964    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Oct 29 23:29:01.913765 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:01.913736    5111 patch_prober.go:29] interesting pod/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Oct 29 23:29:01.913918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:01.913795    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Oct 29 23:29:02.165657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.165620    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:02.771215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.771184    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:29:02.771652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.771581    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:29:02.772268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.772247    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:29:02.920472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.920442    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:29:02.921788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.921766    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:29:02.922623 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:02.922603    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:29:03.790457 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:03.790411    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:05.147646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:05.147610    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:07.678103 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:07.678054    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:07.722093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:07.722050    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:07.768998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:07.768554    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:07.859459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:07.859423    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:07.867079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:07.867046    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.038157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.038058    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.289910 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.289819    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.408117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.408074    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.464117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.464073    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.496119 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.496075    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.526112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.526057    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:08.561144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.561046    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:08.762000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.751320    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:08.763101 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.752860    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:08.763859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.763839    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:08.764735 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.756502    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:08.765378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:08.765361    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:09.513994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.509729    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:09.756999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.755460    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:09.758175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.758143    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:09.807034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.806913    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:09.835567 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.835487    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:09.836229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.836193    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:09.859036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.858242    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:09.859036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.858306    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:09.859036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.858343    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:09.965335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.965295    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:09.982103 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.968622    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:09.982664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.968647    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:09.991012 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:09.984217    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:10.200003 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.197028    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:10.399014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.395741    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:29:10.438804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.438773    5111 cpu_manager.go:414] "containerMap: removing container" podUID="624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" containerName="revoke-keys"
Oct 29 23:29:10.439051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.439032    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" containerName="revoke-keys"
Oct 29 23:29:10.456648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.456543    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" containerName="revoke-keys"
Oct 29 23:29:10.487455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.483736    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:29:10.487455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.483816    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:29:10.512071 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.493309    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.541920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.541727    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:29:10.541920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.541816    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:10.541920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.541852    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:10.541920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.541882    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:10.542250 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.541965    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.558903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.558866    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:10.561368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.559217    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 29 23:29:10.571065 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.564180    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:29:10.571316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.571293    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.581436 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.581403    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:10.598736 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.588057    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:29:10.599134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.594646    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.599264 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599144    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.599264 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599181    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.599264 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599207    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.599264 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599255    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.599500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599278    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.599500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599349    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.599500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599399    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.599500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599425    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599514    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wbrk\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599549    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599588    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgs6z\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599616    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599653    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599732    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599810    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599893    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.609194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.599916    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.629648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.629344    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:29:10.629648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.629400    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:29:10.629648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.629510    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702088    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wbrk\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702154    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702192    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702221    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hjfs\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702268    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgs6z\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702309    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702350    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702375    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702409    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702440    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702497    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702530    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkhq\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702583    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702636    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702684    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702716    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702776    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702813    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702852    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.703045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.702942    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.704668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.703805    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.704668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.704005    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.707401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.707338    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.708240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.708216    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.708948 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.708920    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.709422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.709400    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.709774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.709752    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.710218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.710196    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.710451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.710427    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.710619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.710598    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.712208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712179    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.712335 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712303    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.712410 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712383    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.712472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712431    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.712529 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712511    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.712600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712583    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.712660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712615    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.712717 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712665    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.712785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712742    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.712837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712783    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.712893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712832    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.712893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712858    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.713033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712907    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.713033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.712940    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.713551 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.713518    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.713857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.713836    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.717439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.717396    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.717687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.717665    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.719797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.719771    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.720357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.720334    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.726685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.726656    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.730837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.730809    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgs6z\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-chtns\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.731922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.731880    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wbrk\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.738230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.738202    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\") pod \"e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") " pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.814781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814746    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.814781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814785    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814801    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814843    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkhq\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814870    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814918    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.814943    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815030    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815130    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815182    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815229    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815252    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815310    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815338    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815343    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815364    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815368    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815390    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.815459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815361    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815499    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815529    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hjfs\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815632    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815654    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815692    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815938    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.815955    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.816052    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.816300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.816082    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.817684 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.817659    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.817810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.817774    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.817867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.817825    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.818277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.818261    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.835834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.835805    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.835966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.835906    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hjfs\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:10.836200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.836182    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkhq\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.836257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.836221    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\") pod \"e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") " pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.918453 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.918418    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:10.947706 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.947680    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:10.958232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.958168    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:10.966069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:10.966030    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:11.242012 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.241966    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:29:11.242198 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:11.242175    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1580d4e7_b803_4909_bf1c_317d4bc6224f.slice/crio-011c40d9219a7da3cd9d2bbcc7966664b327c37fa35f41be2e2f9b6813c0848a WatchSource:0}: Error finding container 011c40d9219a7da3cd9d2bbcc7966664b327c37fa35f41be2e2f9b6813c0848a: Status 404 returned error can't find the container with id 011c40d9219a7da3cd9d2bbcc7966664b327c37fa35f41be2e2f9b6813c0848a
Oct 29 23:29:11.244679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.244648    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.244773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.244706    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.244773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.244736    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.282916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.282893    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:29:11.283732 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:11.283708    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309fd91b_71ee_4957_808b_5143e283a5d8.slice/crio-ee6dbdf985d32d545121f29039b8dd1be62f097842bc298d4144115edc71cda1 WatchSource:0}: Error finding container ee6dbdf985d32d545121f29039b8dd1be62f097842bc298d4144115edc71cda1: Status 404 returned error can't find the container with id ee6dbdf985d32d545121f29039b8dd1be62f097842bc298d4144115edc71cda1
Oct 29 23:29:11.286345 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.286315    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.286421 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.286375    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.286421 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.286403    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.319710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.319652    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:29:11.321304 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:11.321262    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45837438_49af_4dea_b0c6_0d0a64649b41.slice/crio-57f1b6734388972103ab3a4bcb527a0d44cea07699963f39f6ab9bd33ad63a9f WatchSource:0}: Error finding container 57f1b6734388972103ab3a4bcb527a0d44cea07699963f39f6ab9bd33ad63a9f: Status 404 returned error can't find the container with id 57f1b6734388972103ab3a4bcb527a0d44cea07699963f39f6ab9bd33ad63a9f
Oct 29 23:29:11.322242 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:11.322218    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded2716a9_be29_4ec9_8ab3_b0b4ffead918.slice/crio-4aebcfa37f26ab7319b438a8b695b4826258e4177f4019977556844b92bfc08c WatchSource:0}: Error finding container 4aebcfa37f26ab7319b438a8b695b4826258e4177f4019977556844b92bfc08c: Status 404 returned error can't find the container with id 4aebcfa37f26ab7319b438a8b695b4826258e4177f4019977556844b92bfc08c
Oct 29 23:29:11.324145 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.324116    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.324245 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.324183    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.324245 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.324215    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.324497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.324472    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:29:11.325118 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.325089    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.325232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.325163    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:11.325232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:11.325211    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:29:12.117336 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.117297    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" event={"ID":"ed2716a9-be29-4ec9-8ab3-b0b4ffead918","Type":"ContainerStarted","Data":"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"}
Oct 29 23:29:12.117867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.117368    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" event={"ID":"ed2716a9-be29-4ec9-8ab3-b0b4ffead918","Type":"ContainerStarted","Data":"4aebcfa37f26ab7319b438a8b695b4826258e4177f4019977556844b92bfc08c"}
Oct 29 23:29:12.119202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.119172    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" event={"ID":"309fd91b-71ee-4957-808b-5143e283a5d8","Type":"ContainerStarted","Data":"5dfa1f60f9f0db1095bab37b70b30cdf3e7a09ed867ed4871170346e1b737b77"}
Oct 29 23:29:12.119386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.119226    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" event={"ID":"309fd91b-71ee-4957-808b-5143e283a5d8","Type":"ContainerStarted","Data":"ee6dbdf985d32d545121f29039b8dd1be62f097842bc298d4144115edc71cda1"}
Oct 29 23:29:12.121210 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.121180    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" event={"ID":"45837438-49af-4dea-b0c6-0d0a64649b41","Type":"ContainerStarted","Data":"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"}
Oct 29 23:29:12.121325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.121230    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" event={"ID":"45837438-49af-4dea-b0c6-0d0a64649b41","Type":"ContainerStarted","Data":"57f1b6734388972103ab3a4bcb527a0d44cea07699963f39f6ab9bd33ad63a9f"}
Oct 29 23:29:12.122749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.122726    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" event={"ID":"1580d4e7-b803-4909-bf1c-317d4bc6224f","Type":"ContainerStarted","Data":"636b53ea06ae1fcefc90a2c2dff7bf2ad940d4438c2873efc321c66c299b18ae"}
Oct 29 23:29:12.122858 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.122768    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" event={"ID":"1580d4e7-b803-4909-bf1c-317d4bc6224f","Type":"ContainerStarted","Data":"011c40d9219a7da3cd9d2bbcc7966664b327c37fa35f41be2e2f9b6813c0848a"}
Oct 29 23:29:12.177897 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.177850    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podStartSLOduration=2.177837651 podStartE2EDuration="2.177837651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:12.174785523 +0000 UTC m=+963.261068684" watchObservedRunningTime="2026-10-29 23:29:12.177837651 +0000 UTC m=+963.264120785"
Oct 29 23:29:12.223907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.223849    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podStartSLOduration=2.223831354 podStartE2EDuration="2.223831354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:12.220346504 +0000 UTC m=+963.306629653" watchObservedRunningTime="2026-10-29 23:29:12.223831354 +0000 UTC m=+963.310114489"
Oct 29 23:29:12.274140 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.273961    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podStartSLOduration=2.273940074 podStartE2EDuration="2.273940074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:12.266651761 +0000 UTC m=+963.352934897" watchObservedRunningTime="2026-10-29 23:29:12.273940074 +0000 UTC m=+963.360223284"
Oct 29 23:29:12.399101 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.398962    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podStartSLOduration=2.398941406 podStartE2EDuration="2.398941406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:12.39563563 +0000 UTC m=+963.481918766" watchObservedRunningTime="2026-10-29 23:29:12.398941406 +0000 UTC m=+963.485224543"
Oct 29 23:29:12.920640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.920591    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:12.921068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.921031    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:12.921243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.921217    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:12.978767 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.978734    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:12.979132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.979106    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:12.979300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.979276    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:12.987070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.987041    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:12.987248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.987225    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:12.988276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.988257    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:12.988447 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:12.988435    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:13.008388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.008350    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:13.008703 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.008674    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:13.460158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.460098    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:13.503536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.503490    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:13.544830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.541819    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:13.545139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.541873    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:13.828058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.827095    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:13.930190 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.929741    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:13.930190 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.929799    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:13.950395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.950366    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:13.950649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.950624    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:13.959374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.959279    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:13.959374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.959323    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:13.969762 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.969739    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:13.969935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:13.969911    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:14.167626 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.167537    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.248086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.248045    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:14.463144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.463049    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.542120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.542070    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.542747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.542370    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.543875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.543579    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.544665 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.543607    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:14.545428 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.544395    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:29:14.553016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.545944    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.566323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.557089    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:14.566569 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.566553    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:14.862999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.830441    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:14.862999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.830834    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:15.039115 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:14.997342    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 29 23:29:15.039115 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.004118    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:15.039115 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.004180    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:15.084000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.083089    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:15.084000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.083150    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:15.084000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.083233    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:15.084000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.083257    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:15.113757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.113651    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:15.140084 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.140041    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:15.173007 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.171037    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.232999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.225829    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:15.294064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.277181    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:15.319925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.318292    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.388241 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.388154    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:15.468005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.459935    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.520285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.520252    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.545388    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.546013    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.547024    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.548199    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.548260    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.548295    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:15.595998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.592780    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:15.628707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.616150    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:29:15.809048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.804089    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:15.991956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.990709    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:15.991956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.990769    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:16.009010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:15.995437    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.026779    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.027112    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.029009    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.029068    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.029330    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:16.034998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.029362    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:16.058417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.058345    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:16.062209 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.058674    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:16.204094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.204053    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.419034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.409473    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.419034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.410119    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:16.419034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.411260    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.565997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.555264    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.595063 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.583177    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:16.598385 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.596089    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:16.598385 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.597224    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:16.599205 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.598638    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:16.600481 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.599352    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:16.618940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.616031    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.618940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.618487    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.817868 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.817628    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:16.818084 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.818018    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:29:16.851720 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:16.847474    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.015998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.013341    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.015998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.013395    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:17.015998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.013766    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.015998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.013801    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:17.034822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.027114    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.034822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.027167    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:17.034822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.027246    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.034822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.027271    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:17.330999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.330198    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.350000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.331379    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.434005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.432531    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.496000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.477329    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.528100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.527327    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:17.623911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.614555    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.645994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.645928    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.646419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.646303    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.791156 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.774104    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.822743 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.819232    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.843040 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.819304    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.843040 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.819370    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.843040 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.819392    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.846822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.819099    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:17.860017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.859046    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:17.948896 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.948812    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.949183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.949153    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:17.949412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.949385    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:17.949550 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:17.949524    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:18.101866 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.101742    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:18.102324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.102194    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:18.105296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.105270    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:18.124899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.124872    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:18.125198 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.124705    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.125421 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.110940    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:18.125708 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.125694    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:18.340640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.340597    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.408030 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.352311    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.438425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.437074    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.482098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.482054    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.501006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.483089    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:18.639073 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.633097    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.681055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.651370    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.682908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.681736    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:18.818108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.818059    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.872144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.872099    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.896053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.874030    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:18.896651 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.896614    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.897142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.897115    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.897556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.897528    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.897901 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.897879    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:18.946018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.924729    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:18.946018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.924790    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:18.948105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.948076    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:18.948294 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.948270    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:18.958517 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.958491    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:18.959564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.959540    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:18.975088 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:18.975043    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:19.008999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.000233    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:19.008999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.000278    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:19.229286 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.226060    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": context deadline exceeded"
Oct 29 23:29:19.269614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.269580    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:19.339223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.339195    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:19.343666 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.343644    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:19.345929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.345904    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:29:19.360248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.360223    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:19.374208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.374188    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:19.387441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.387422    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:19.390502 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.390483    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:19.448130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.448092    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:19.472500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.470252    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:19.476870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.474476    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:19.503665 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.503588    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:19.509214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.509188    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:19.738998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.738414    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:19.919669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.919267    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Oct 29 23:29:19.919669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.919332    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Oct 29 23:29:19.948486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.948378    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Oct 29 23:29:19.948486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.948430    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Oct 29 23:29:19.960619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.958631    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Oct 29 23:29:19.960619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.958679    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Oct 29 23:29:19.967205 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.967173    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Oct 29 23:29:19.967339 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:19.967252    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Oct 29 23:29:21.035114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.035054    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:21.036368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.036347    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:21.039225 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.039203    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:29:21.048816 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.048791    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:21.049497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.049474    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:21.050381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.050360    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:29:21.299562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.299409    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:21.300908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.300888    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:21.301770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.301699    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:29:21.450171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.446993    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:21.450171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.448617    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:21.450171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.449966    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:29:21.609493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:21.609456    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:23.172830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.172795    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:23.280718 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.280684    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"]
Oct 29 23:29:23.290225 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.290197    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.299030 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.298998    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"]
Oct 29 23:29:23.303338 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.303297    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.303464 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.303414    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjm5z\" (UniqueName: \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.303530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.303470    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.308185 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.308165    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:29:23.308536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.308517    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:29:23.319810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.319768    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:29:23.319810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.319802    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:29:23.404935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.404905    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.405156 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.405037    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjm5z\" (UniqueName: \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.405156 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.405096    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.405566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.405545    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.407612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.407586    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.426488 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.426424    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjm5z\" (UniqueName: \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\") pod \"maas-api-revoke-sub-975f1321-9cc66\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.614965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.614934    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:23.930159 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:23.928760    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"]
Oct 29 23:29:23.932895 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:23.932864    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adbb6cb_4b92_4430_983b_40f64cce4ecc.slice/crio-5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929 WatchSource:0}: Error finding container 5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929: Status 404 returned error can't find the container with id 5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929
Oct 29 23:29:24.301472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:24.301428    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66" event={"ID":"4adbb6cb-4b92-4430-983b-40f64cce4ecc","Type":"ContainerStarted","Data":"5e29eed8c53268cb32cb3cf206f347fdf5680f2f6fc45445fdcdabbc3c354a3c"}
Oct 29 23:29:24.301838 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:24.301496    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66" event={"ID":"4adbb6cb-4b92-4430-983b-40f64cce4ecc","Type":"ContainerStarted","Data":"5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929"}
Oct 29 23:29:24.345334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:24.345290    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66" podStartSLOduration=1.3452769070000001 podStartE2EDuration="1.345276907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:24.342059715 +0000 UTC m=+975.428342850" watchObservedRunningTime="2026-10-29 23:29:24.345276907 +0000 UTC m=+975.431560040"
Oct 29 23:29:24.774313 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:24.774281    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:25.309011 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:25.308965    5111 generic.go:356] "Generic (PLEG): container finished" podID="4adbb6cb-4b92-4430-983b-40f64cce4ecc" containerID="5e29eed8c53268cb32cb3cf206f347fdf5680f2f6fc45445fdcdabbc3c354a3c" exitCode=0
Oct 29 23:29:25.309461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:25.309066    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66" event={"ID":"4adbb6cb-4b92-4430-983b-40f64cce4ecc","Type":"ContainerDied","Data":"5e29eed8c53268cb32cb3cf206f347fdf5680f2f6fc45445fdcdabbc3c354a3c"}
Oct 29 23:29:26.147931 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:29:26.147897    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:29:26.180061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.180028    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:26.341174 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.339005    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:26.522664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.522638    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:26.549804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.549777    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\") pod \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") "
Oct 29 23:29:26.549918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.549870    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\" (UniqueName: \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\") pod \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") "
Oct 29 23:29:26.550033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.550014    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\") pod \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\" (UID: \"4adbb6cb-4b92-4430-983b-40f64cce4ecc\") "
Oct 29 23:29:26.550430 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.550394    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca" pod "4adbb6cb-4b92-4430-983b-40f64cce4ecc" (UID: "4adbb6cb-4b92-4430-983b-40f64cce4ecc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:26.550632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.550612    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:26.551818 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.551792    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca" pod "4adbb6cb-4b92-4430-983b-40f64cce4ecc" (UID: "4adbb6cb-4b92-4430-983b-40f64cce4ecc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:26.551941 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.551922    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z" pod "4adbb6cb-4b92-4430-983b-40f64cce4ecc" (UID: "4adbb6cb-4b92-4430-983b-40f64cce4ecc"). InnerVolumeSpecName "kube-api-access-jjm5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:26.651745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.651715    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjm5z\" (UniqueName: \"kubernetes.io/projected/4adbb6cb-4b92-4430-983b-40f64cce4ecc-kube-api-access-jjm5z\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:26.651745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.651744    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/4adbb6cb-4b92-4430-983b-40f64cce4ecc-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:26.731242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.731215    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"]
Oct 29 23:29:26.731757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.731744    5111 cpu_manager.go:414] "containerMap: removing container" podUID="4adbb6cb-4b92-4430-983b-40f64cce4ecc" containerName="revoke-keys"
Oct 29 23:29:26.731800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.731759    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4adbb6cb-4b92-4430-983b-40f64cce4ecc" containerName="revoke-keys"
Oct 29 23:29:26.731860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.731851    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="4adbb6cb-4b92-4430-983b-40f64cce4ecc" containerName="revoke-keys"
Oct 29 23:29:26.735233 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.735216    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.756780 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.756743    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"]
Oct 29 23:29:26.853628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.853598    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6ldc\" (UniqueName: \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.853770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.853670    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.853770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.853754    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.955358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.955330    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6ldc\" (UniqueName: \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.955508 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.955379    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.955508 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.955418    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.956094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.956071    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.957848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.957817    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:26.976080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:26.976060    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6ldc\" (UniqueName: \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\") pod \"maas-api-revoke-sub-b47fbb16-f28ms\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:27.052412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:27.052360    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:27.324643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:27.324621    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"
Oct 29 23:29:27.324797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:27.324620    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66" event={"ID":"4adbb6cb-4b92-4430-983b-40f64cce4ecc","Type":"ContainerDied","Data":"5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929"}
Oct 29 23:29:27.324797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:27.324741    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c87f435b41ea2cdd08950ac3ad68f41d18a52ff0d062c9152ff998aa260d929"
Oct 29 23:29:27.355644 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:27.355618    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae70ecd5_bbde_4d76_b89e_f955b1bc3abb.slice/crio-82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef WatchSource:0}: Error finding container 82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef: Status 404 returned error can't find the container with id 82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef
Oct 29 23:29:27.355644 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:27.355634    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"]
Oct 29 23:29:28.099517 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:28.099482    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:28.342086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:28.342054    5111 generic.go:356] "Generic (PLEG): container finished" podID="ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" containerID="66d1a82f34210351304cf809a1ffb495d2e3c1aaa31c681a444712fc5ae74680" exitCode=0
Oct 29 23:29:28.342213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:28.342095    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms" event={"ID":"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb","Type":"ContainerDied","Data":"66d1a82f34210351304cf809a1ffb495d2e3c1aaa31c681a444712fc5ae74680"}
Oct 29 23:29:28.342213 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:28.342167    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms" event={"ID":"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb","Type":"ContainerStarted","Data":"82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef"}
Oct 29 23:29:29.515697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.515674    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:29.587942 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.587910    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\") pod \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") "
Oct 29 23:29:29.588405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.588377    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca" pod "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" (UID: "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:29.588526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.588108    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\") pod \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") "
Oct 29 23:29:29.588616 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.588597    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\" (UniqueName: \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\") pod \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\" (UID: \"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb\") "
Oct 29 23:29:29.589334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.589311    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:29.592135 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.592094    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca" pod "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" (UID: "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:29.610064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.609657    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc" pod "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" (UID: "ae70ecd5-bbde-4d76-b89e-f955b1bc3abb"). InnerVolumeSpecName "kube-api-access-x6ldc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:29.619364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.619262    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:29.690693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.690656    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:29.690693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:29.690692    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6ldc\" (UniqueName: \"kubernetes.io/projected/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb-kube-api-access-x6ldc\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:30.357039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:30.357006    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms" event={"ID":"ae70ecd5-bbde-4d76-b89e-f955b1bc3abb","Type":"ContainerDied","Data":"82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef"}
Oct 29 23:29:30.357200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:30.357059    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82f3c96e48eae813d352db1f029cfcba1b77a3c35f8bc6daa165182ec4c312ef"
Oct 29 23:29:30.357200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:30.357014    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"
Oct 29 23:29:31.054725 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:31.054670    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:31.158454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:31.158420    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:29:31.174776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:31.174737    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:29:31.301872 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:31.301839    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:32.908999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:32.899477    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:33.830000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:33.829443    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:33.847998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:33.845073    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:33.847998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:33.845993    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:33.986999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:33.964184    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:34.264840 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.264086    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.348308 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.347079    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.354120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.354049    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.382122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.382069    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.395089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.395048    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.443046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.442617    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.475447 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.474131    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.475447 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.475315    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:34.475447 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.475362    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.626139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.620080    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:29:34.685124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.685081    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:34.685389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.685358    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.727418 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.712362    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:34.845372 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.845330    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:34.986120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.931594    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:34.986120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.972117    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:34.986120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.972225    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.986120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.975559    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:34.986120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:34.976708    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:35.524000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.503786    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:35.593997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.527615    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 29 23:29:35.593997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.528080    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.621018 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.620957    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.669005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.662916    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:35.703505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.703459    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.811174 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.811075    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:35.813775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.813737    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.847998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.825668    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.848535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.845299    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": context deadline exceeded"
Oct 29 23:29:35.958507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.958457    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:35.971122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:35.971081    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": context deadline exceeded"
Oct 29 23:29:36.054999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.029796    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:36.056781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.056742    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:36.057006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.056988    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:36.057176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.057162    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:36.057317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.057305    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:36.061076 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.061052    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:36.116091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.091614    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:36.194105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.194074    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:36.194357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.194338    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:36.195337 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.106715    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:36.603999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.600176    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.620087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.620049    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.841872 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.624536    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:36.842388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.690098    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.843588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.691310    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.819257    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.844402    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.819310    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/health\": context deadline exceeded"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.819291    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.820476    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.839086    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:36.845261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.839118    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:29:36.884077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:36.863988    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.030337 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.009376    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:37.056044 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.032180    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.145304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.145089    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:37.145304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.145161    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.202005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.196838    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.388080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.379308    5111 patch_prober.go:29] interesting pod/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:37.388080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.379366    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.389146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.389067    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.437256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.437182    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:37.619771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.619718    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.721080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.720991    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:37.726001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.722517    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:37.750368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.742239    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:37.832831 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.832803    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:37.841789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.841755    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.863268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.863231    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.882397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.868357    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:37.882397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.868386    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 29 23:29:37.882397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.868406    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:37.922022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.914247    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:37.956885 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.956821    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:37.997470 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.997396    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:37.999390 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:37.999369    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:38.040343 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.040319    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:38.049217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.049190    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:38.052922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.052890    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:38.069526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.069504    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:38.089659 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.089617    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:38.117085 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:38.117021    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:39.574068 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:39.574032    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:41.341373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.341342    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"]
Oct 29 23:29:41.341940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.341926    5111 cpu_manager.go:414] "containerMap: removing container" podUID="ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" containerName="revoke-keys"
Oct 29 23:29:41.342041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.341942    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" containerName="revoke-keys"
Oct 29 23:29:41.342077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.342059    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" containerName="revoke-keys"
Oct 29 23:29:41.345789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.345765    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.365255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.365235    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"]
Oct 29 23:29:41.375093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.375028    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:29:41.375443 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.375300    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:29:41.375541 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.375441    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:29:41.375905 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.375890    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:29:41.407830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.407799    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:41.499406 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.499365    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbfwr\" (UniqueName: \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.499724 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.499693    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.499936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.499893    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.601525 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.601449    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbfwr\" (UniqueName: \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.601525 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.601509    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.601755 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.601564    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.602268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.602240    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.603958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.603936    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.622307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.622285    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbfwr\" (UniqueName: \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\") pod \"maas-api-revoke-sub-6566c8f5-zgwkp\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.662339 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.662317    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:41.965091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:41.965063    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"]
Oct 29 23:29:41.966330 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:41.966304    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5088088_4a8f_4ef0_9af9_65325f8b03c3.slice/crio-6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8 WatchSource:0}: Error finding container 6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8: Status 404 returned error can't find the container with id 6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8
Oct 29 23:29:42.451738 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.451707    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp" event={"ID":"f5088088-4a8f-4ef0-9af9-65325f8b03c3","Type":"ContainerStarted","Data":"fba922a25803f3a17241128cd927746b7ac5630a423f4893d5386c08b9270007"}
Oct 29 23:29:42.452122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.451767    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp" event={"ID":"f5088088-4a8f-4ef0-9af9-65325f8b03c3","Type":"ContainerStarted","Data":"6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8"}
Oct 29 23:29:42.494837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.494797    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp" podStartSLOduration=1.494783123 podStartE2EDuration="1.494783123s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:42.493745941 +0000 UTC m=+993.580029075" watchObservedRunningTime="2026-10-29 23:29:42.494783123 +0000 UTC m=+993.581066256"
Oct 29 23:29:42.534708 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.534641    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"]
Oct 29 23:29:42.539093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.539069    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.560150 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.560128    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"]
Oct 29 23:29:42.613497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.613462    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.613657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.613636    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.613720 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.613674    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf4t9\" (UniqueName: \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.714441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.714411    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.714566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.714493    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.714566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.714513    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf4t9\" (UniqueName: \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.715030 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.715010    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.716799 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.716755    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.734962 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.734935    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf4t9\" (UniqueName: \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\") pod \"maas-api-revoke-sub-663037f8-bnlt8\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.853575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.853545    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:42.878194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:42.878167    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:43.164249 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:43.164215    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad92f618_7f98_4939_80da_3e649335cc67.slice/crio-04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f WatchSource:0}: Error finding container 04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f: Status 404 returned error can't find the container with id 04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f
Oct 29 23:29:43.165549 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:43.165523    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"]
Oct 29 23:29:43.460248 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:43.460177    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8" event={"ID":"ad92f618-7f98-4939-80da-3e649335cc67","Type":"ContainerStarted","Data":"04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f"}
Oct 29 23:29:43.462023 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:43.461995    5111 generic.go:356] "Generic (PLEG): container finished" podID="f5088088-4a8f-4ef0-9af9-65325f8b03c3" containerID="fba922a25803f3a17241128cd927746b7ac5630a423f4893d5386c08b9270007" exitCode=0
Oct 29 23:29:43.462153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:43.462034    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp" event={"ID":"f5088088-4a8f-4ef0-9af9-65325f8b03c3","Type":"ContainerDied","Data":"fba922a25803f3a17241128cd927746b7ac5630a423f4893d5386c08b9270007"}
Oct 29 23:29:44.439295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.439265    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:44.478004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.477955    5111 generic.go:356] "Generic (PLEG): container finished" podID="ad92f618-7f98-4939-80da-3e649335cc67" containerID="6164f3289dbf222ffd247644827b890367cd780fcf14cdade804af1a73fd8845" exitCode=0
Oct 29 23:29:44.478540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.478093    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8" event={"ID":"ad92f618-7f98-4939-80da-3e649335cc67","Type":"ContainerDied","Data":"6164f3289dbf222ffd247644827b890367cd780fcf14cdade804af1a73fd8845"}
Oct 29 23:29:44.633584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.633562    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:44.740083 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.740014    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\") pod \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") "
Oct 29 23:29:44.740220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.740202    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\") pod \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") "
Oct 29 23:29:44.740322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.740308    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\" (UniqueName: \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\") pod \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\" (UID: \"f5088088-4a8f-4ef0-9af9-65325f8b03c3\") "
Oct 29 23:29:44.740383 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.740330    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca" pod "f5088088-4a8f-4ef0-9af9-65325f8b03c3" (UID: "f5088088-4a8f-4ef0-9af9-65325f8b03c3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:44.740804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.740765    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:44.742365 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.742340    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr" pod "f5088088-4a8f-4ef0-9af9-65325f8b03c3" (UID: "f5088088-4a8f-4ef0-9af9-65325f8b03c3"). InnerVolumeSpecName "kube-api-access-qbfwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:44.742451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.742363    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca" pod "f5088088-4a8f-4ef0-9af9-65325f8b03c3" (UID: "f5088088-4a8f-4ef0-9af9-65325f8b03c3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:44.841854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.841834    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbfwr\" (UniqueName: \"kubernetes.io/projected/f5088088-4a8f-4ef0-9af9-65325f8b03c3-kube-api-access-qbfwr\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:44.841854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:44.841855    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5088088-4a8f-4ef0-9af9-65325f8b03c3-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:45.486045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.486010    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp" event={"ID":"f5088088-4a8f-4ef0-9af9-65325f8b03c3","Type":"ContainerDied","Data":"6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8"}
Oct 29 23:29:45.486428 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.486085    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ed01f94298cca3292b7544ffde520e62d0d9b2d66a185551e4c1a9062676ec8"
Oct 29 23:29:45.486428 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.486018    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"
Oct 29 23:29:45.641715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.641687    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:45.750870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.750797    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\") pod \"ad92f618-7f98-4939-80da-3e649335cc67\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") "
Oct 29 23:29:45.750870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.750845    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\" (UniqueName: \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\") pod \"ad92f618-7f98-4939-80da-3e649335cc67\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") "
Oct 29 23:29:45.751676 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.751088    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\") pod \"ad92f618-7f98-4939-80da-3e649335cc67\" (UID: \"ad92f618-7f98-4939-80da-3e649335cc67\") "
Oct 29 23:29:45.751676 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.751211    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca" pod "ad92f618-7f98-4939-80da-3e649335cc67" (UID: "ad92f618-7f98-4939-80da-3e649335cc67"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:45.751676 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.751535    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:45.752948 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.752924    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9" pod "ad92f618-7f98-4939-80da-3e649335cc67" (UID: "ad92f618-7f98-4939-80da-3e649335cc67"). InnerVolumeSpecName "kube-api-access-zf4t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:45.753100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.752994    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca" pod "ad92f618-7f98-4939-80da-3e649335cc67" (UID: "ad92f618-7f98-4939-80da-3e649335cc67"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:45.853069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.853038    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zf4t9\" (UniqueName: \"kubernetes.io/projected/ad92f618-7f98-4939-80da-3e649335cc67-kube-api-access-zf4t9\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:45.853069 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.853070    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ad92f618-7f98-4939-80da-3e649335cc67-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:45.907026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:45.906993    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:46.189886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.189851    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" containerID="cri-o://b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a" gracePeriod=45
Oct 29 23:29:46.203711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.203676    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" containerID="cri-o://4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39" gracePeriod=45
Oct 29 23:29:46.344360 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.344313    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": dial tcp 10.132.0.53:8000: connect: connection refused"
Oct 29 23:29:46.466756 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.466730    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:46.488368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.488342    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:46.493081 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.493061    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"
Oct 29 23:29:46.493212 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.493105    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8" event={"ID":"ad92f618-7f98-4939-80da-3e649335cc67","Type":"ContainerDied","Data":"04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f"}
Oct 29 23:29:46.493212 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.493152    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="04f1867396cba2b4c312a4480d47a2ec228fd84f57e02f87110da0054afdee3f"
Oct 29 23:29:46.494501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.494479    5111 generic.go:356] "Generic (PLEG): container finished" podID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerID="4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39" exitCode=0
Oct 29 23:29:46.494568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.494521    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"
Oct 29 23:29:46.494631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.494557    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" event={"ID":"b4d66b98-2026-49a8-8f30-3fd0ee85b20c","Type":"ContainerDied","Data":"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"}
Oct 29 23:29:46.494631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.494599    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q" event={"ID":"b4d66b98-2026-49a8-8f30-3fd0ee85b20c","Type":"ContainerDied","Data":"bc6e35f87ce66a7bb88aa8d7538eb4a549c4f98ad268baa1b37756e30a4746d2"}
Oct 29 23:29:46.494631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.494619    5111 scope.go:122] "RemoveContainer" containerID="4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"
Oct 29 23:29:46.496059 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.496037    5111 generic.go:356] "Generic (PLEG): container finished" podID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerID="b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a" exitCode=0
Oct 29 23:29:46.496153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.496071    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" event={"ID":"e24f630f-ccc6-432b-8ff9-636d959f2980","Type":"ContainerDied","Data":"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"}
Oct 29 23:29:46.496153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.496081    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"
Oct 29 23:29:46.496153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.496090    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" event={"ID":"e24f630f-ccc6-432b-8ff9-636d959f2980","Type":"ContainerDied","Data":"43b30217faefbee23bf1629aae12b93ad5e9d96ed1919acdee19e895ca349253"}
Oct 29 23:29:46.504236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.504220    5111 scope.go:122] "RemoveContainer" containerID="4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"
Oct 29 23:29:46.504486 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:29:46.504469    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39\": container with ID starting with 4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39 not found: ID does not exist" containerID="4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"
Oct 29 23:29:46.504522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.504494    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39"} err="failed to get container status \"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39\": rpc error: code = NotFound desc = could not find container \"4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39\": container with ID starting with 4c726d0615baf4eb795813c8209f42902de1bfe78473e0efba8fb7b1de7a3d39 not found: ID does not exist"
Oct 29 23:29:46.504522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.504509    5111 scope.go:122] "RemoveContainer" containerID="b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"
Oct 29 23:29:46.515225 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.515176    5111 scope.go:122] "RemoveContainer" containerID="b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"
Oct 29 23:29:46.515516 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:29:46.515493    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a\": container with ID starting with b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a not found: ID does not exist" containerID="b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"
Oct 29 23:29:46.515578 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.515526    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a"} err="failed to get container status \"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a\": rpc error: code = NotFound desc = could not find container \"b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a\": container with ID starting with b6464c65ff9fa593cfafc93a200c729d28fc9bcc41df12d131e280f869e8d41a not found: ID does not exist"
Oct 29 23:29:46.563019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.562988    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563083    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.563184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563119    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.563184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563153    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563173    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.563415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563234    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.563415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563294    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\" (UniqueName: \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.563415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563363    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.563566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563469    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\" (UniqueName: \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563508    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\" (UniqueName: \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563539    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563588    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.563715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563593    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\") pod \"e24f630f-ccc6-432b-8ff9-636d959f2980\" (UID: \"e24f630f-ccc6-432b-8ff9-636d959f2980\") "
Oct 29 23:29:46.563817 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563733    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\") pod \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\" (UID: \"b4d66b98-2026-49a8-8f30-3fd0ee85b20c\") "
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.563935    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564048    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564322    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564586    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564611    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564626    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564639    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.564957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.564652    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.566051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.565781    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.566051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.565806    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.567824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.567788    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:46.567948 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.567904    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:46.568342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.568307    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:29:46.568650 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.568617    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2" pod "b4d66b98-2026-49a8-8f30-3fd0ee85b20c" (UID: "b4d66b98-2026-49a8-8f30-3fd0ee85b20c"). InnerVolumeSpecName "kube-api-access-hwnh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:46.569314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.569286    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb" pod "e24f630f-ccc6-432b-8ff9-636d959f2980" (UID: "e24f630f-ccc6-432b-8ff9-636d959f2980"). InnerVolumeSpecName "kube-api-access-qj9tb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:46.666168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666141    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666168 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666168    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwnh2\" (UniqueName: \"kubernetes.io/projected/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-kube-api-access-hwnh2\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666178    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e24f630f-ccc6-432b-8ff9-636d959f2980-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666189    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj9tb\" (UniqueName: \"kubernetes.io/projected/e24f630f-ccc6-432b-8ff9-636d959f2980-kube-api-access-qj9tb\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666201    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666209    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e24f630f-ccc6-432b-8ff9-636d959f2980-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.666323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.666217    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d66b98-2026-49a8-8f30-3fd0ee85b20c-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:46.847472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.847440    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:29:46.854537 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.854509    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/premium-w1-313d9c-kserve-5cb9b9db7b-2gg7q"]
Oct 29 23:29:46.889051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.889023    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:29:46.892262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:46.892241    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp"]
Oct 29 23:29:47.063183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.063117    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" path="/var/lib/kubelet/pods/b4d66b98-2026-49a8-8f30-3fd0ee85b20c/volumes"
Oct 29 23:29:47.063628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.063605    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" path="/var/lib/kubelet/pods/e24f630f-ccc6-432b-8ff9-636d959f2980/volumes"
Oct 29 23:29:47.406136 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.406102    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:47.434072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.434028    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-313d9c/distinct-w1-313d9c-kserve-7ff4466df5-plrnp" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded"
Oct 29 23:29:47.810292 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.810209    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"]
Oct 29 23:29:47.810993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.810956    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f5088088-4a8f-4ef0-9af9-65325f8b03c3" containerName="revoke-keys"
Oct 29 23:29:47.810993 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.810993    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5088088-4a8f-4ef0-9af9-65325f8b03c3" containerName="revoke-keys"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811014    5111 cpu_manager.go:414] "containerMap: removing container" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811021    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811043    5111 cpu_manager.go:414] "containerMap: removing container" podUID="ad92f618-7f98-4939-80da-3e649335cc67" containerName="revoke-keys"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811051    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad92f618-7f98-4939-80da-3e649335cc67" containerName="revoke-keys"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811098    5111 cpu_manager.go:414] "containerMap: removing container" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main"
Oct 29 23:29:47.811141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811105    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main"
Oct 29 23:29:47.811440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811264    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4d66b98-2026-49a8-8f30-3fd0ee85b20c" containerName="main"
Oct 29 23:29:47.811440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811284    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5088088-4a8f-4ef0-9af9-65325f8b03c3" containerName="revoke-keys"
Oct 29 23:29:47.811440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811294    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="e24f630f-ccc6-432b-8ff9-636d959f2980" containerName="main"
Oct 29 23:29:47.811440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.811307    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad92f618-7f98-4939-80da-3e649335cc67" containerName="revoke-keys"
Oct 29 23:29:47.816509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.816487    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.833091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.833068    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"]
Oct 29 23:29:47.842102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.842075    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:29:47.842102 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.842094    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:29:47.844295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.842155    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:29:47.855034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.845967    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:29:47.880656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.880627    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.880753 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.880666    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhjm5\" (UniqueName: \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.880922 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.880901    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.982640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.982610    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.982758 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.982739    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.982832 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.982782    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhjm5\" (UniqueName: \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.983402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.983381    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:47.984953 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:47.984931    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:48.003779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:48.003754    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhjm5\" (UniqueName: \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\") pod \"maas-api-revoke-sub-114fa85f-m59j4\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:48.130945 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:48.130903    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:48.451008 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:48.450964    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476ad2fc_49fb_4f64_ac9a_e18f154d2dcc.slice/crio-5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4 WatchSource:0}: Error finding container 5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4: Status 404 returned error can't find the container with id 5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4
Oct 29 23:29:48.453530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:48.453498    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"]
Oct 29 23:29:48.529039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:48.529003    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4" event={"ID":"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc","Type":"ContainerStarted","Data":"5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4"}
Oct 29 23:29:48.783342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:48.783313    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:49.536890 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:49.536861    5111 generic.go:356] "Generic (PLEG): container finished" podID="476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" containerID="563a58429e0b028eb629a72d7124d9a0fca1ad772cb1b8029ad5b19b5533d46d" exitCode=0
Oct 29 23:29:49.537292 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:49.536950    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4" event={"ID":"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc","Type":"ContainerDied","Data":"563a58429e0b028eb629a72d7124d9a0fca1ad772cb1b8029ad5b19b5533d46d"}
Oct 29 23:29:50.198930 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:50.198842    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:51.518998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.501394    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:51.534996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.532433    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:51.562009 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.559736    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\") pod \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") "
Oct 29 23:29:51.678004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.669368    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca" pod "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" (UID: "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:51.742004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.708123    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5" pod "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" (UID: "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc"). InnerVolumeSpecName "kube-api-access-dhjm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:51.743071 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.722386    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\" (UniqueName: \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\") pod \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") "
Oct 29 23:29:51.743293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.743273    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\") pod \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\" (UID: \"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc\") "
Oct 29 23:29:51.766995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.744114    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhjm5\" (UniqueName: \"kubernetes.io/projected/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-kube-api-access-dhjm5\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:51.791054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.791025    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:51.816381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.816349    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca" pod "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" (UID: "476ad2fc-49fb-4f64-ac9a-e18f154d2dcc"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:51.845815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.845778    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded"
Oct 29 23:29:51.869229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.869177    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:51.870256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.870221    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:51.903024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.880840    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:51.903926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.903499    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4" event={"ID":"476ad2fc-49fb-4f64-ac9a-e18f154d2dcc","Type":"ContainerDied","Data":"5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4"}
Oct 29 23:29:51.903926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.903559    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b800387891de928286d8a657a8a66a1f45bfaff1c7d392dc1d3cb57ed72a8d4"
Oct 29 23:29:51.903926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.903628    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"
Oct 29 23:29:51.923178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.923109    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:51.923589 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:51.923560    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.041334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.031062    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.111446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.105027    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.121776 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.104964    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.166010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.161062    5111 patch_prober.go:29] interesting pod/e2e-worker-w6-837245-openshift-default-67857c7775-chtns container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:29:52.166010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.161136    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.490998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.490064    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/health\": context deadline exceeded"
Oct 29 23:29:52.848925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.848865    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:52.897000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.892267    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.897000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.892629    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.897000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.893264    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:52.897000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.893319    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:52.897000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.894943    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.943046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.919653    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.943046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.920021    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:52.943046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.929042    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:52.943046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:52.929372    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:53.058141 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.057809    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.070205 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.067267    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:53.129055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.127844    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.176999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.158025    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": context deadline exceeded"
Oct 29 23:29:53.253146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.253102    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.463262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.463183    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:53.463523 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.463505    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:53.901000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.867330    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.928000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.903137    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.929375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.908068    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": context deadline exceeded"
Oct 29 23:29:53.950455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.910195    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.950662 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.910251    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Oct 29 23:29:53.954064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.922049    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:53.994024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.951794    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:53.994024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.957138    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:53.994024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:53.952126    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": context deadline exceeded"
Oct 29 23:29:54.131133 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.131082    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:54.161139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.159919    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:54.203116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.203071    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:54.291053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.274708    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:54.296160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.296128    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:54.475221 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.475076    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded"
Oct 29 23:29:54.899089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.899046    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:54.911108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.909087    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:29:54.974119 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.974073    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:54.974668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:54.974637    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:55.006033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.000398    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:55.009826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.009795    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" probeResult="failure" output="Get \"https://10.132.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:55.022021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.010736    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:29:55.022021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.020343    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:55.375054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.291921    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:29:55.375054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.295809    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:29:55.375054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.351177    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" probeResult="failure" output="Get \"https://10.132.0.44:8000/ready\": context deadline exceeded"
Oct 29 23:29:55.414328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.352534    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" probeResult="failure" output="Get \"https://10.132.0.48:8000/ready\": context deadline exceeded"
Oct 29 23:29:55.505165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.502473    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:29:55.505165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.502542    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:29:55.528203 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.528177    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:29:55.536564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.536537    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:29:55.538058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.538033    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:29:55.571510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.571486    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:29:55.581449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.574859    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:29:55.583639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.583613    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:29:55.586039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.585961    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:29:55.589259 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.589242    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:29:55.811132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.811064    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"]
Oct 29 23:29:55.811619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.811607    5111 cpu_manager.go:414] "containerMap: removing container" podUID="476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" containerName="revoke-keys"
Oct 29 23:29:55.811663 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.811621    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" containerName="revoke-keys"
Oct 29 23:29:55.811727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.811719    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" containerName="revoke-keys"
Oct 29 23:29:55.815796 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.815774    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:55.839992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.838399    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"]
Oct 29 23:29:55.839992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.839405    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:29:55.839992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.839890    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:29:55.840530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.840508    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:29:55.840639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.840617    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:29:55.914490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.914456    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d4cp\" (UniqueName: \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:55.914917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.914518    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:55.914917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:55.914735    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.015504 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.015472    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d4cp\" (UniqueName: \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.015638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.015518    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.015638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.015633    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.016312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.016283    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.017849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.017819    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.036564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.036537    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d4cp\" (UniqueName: \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\") pod \"maas-api-revoke-sub-b1870970-r42vm\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.130612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.130587    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:56.154835 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:29:56.154803    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:29:56.439219 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.439149    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:29:56.445079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.445051    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"]
Oct 29 23:29:56.448884 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:56.448855    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e555453_98e5_4aa4_97da_2a647289beba.slice/crio-158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2 WatchSource:0}: Error finding container 158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2: Status 404 returned error can't find the container with id 158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2
Oct 29 23:29:56.461121 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.461092    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:29:56.588537 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.588511    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:56.606687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:56.606651    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm" event={"ID":"1e555453-98e5-4aa4-97da-2a647289beba","Type":"ContainerStarted","Data":"158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2"}
Oct 29 23:29:57.613936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.613909    5111 generic.go:356] "Generic (PLEG): container finished" podID="1e555453-98e5-4aa4-97da-2a647289beba" containerID="a94985f74849d8bdc7210ca8ebccffd547d5c743c08559d02d52cd9e7c54438c" exitCode=0
Oct 29 23:29:57.614345 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.614000    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm" event={"ID":"1e555453-98e5-4aa4-97da-2a647289beba","Type":"ContainerDied","Data":"a94985f74849d8bdc7210ca8ebccffd547d5c743c08559d02d52cd9e7c54438c"}
Oct 29 23:29:57.776444 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.776387    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"]
Oct 29 23:29:57.780470 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.780453    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.800239 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.800213    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"]
Oct 29 23:29:57.837267 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.837241    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.837406 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.837371    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnwsv\" (UniqueName: \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.837475 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.837418    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.938794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.938758    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.938934 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.938913    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.939124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.939091    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnwsv\" (UniqueName: \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.939918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.939891    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.941697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.941666    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:57.965566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:57.965536    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnwsv\" (UniqueName: \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\") pod \"maas-api-revoke-sub-b5f82696-fvfd4\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:58.036079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.035921    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:58.093753 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.093719    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:29:58.408721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.408688    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"]
Oct 29 23:29:58.409170 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:29:58.409140    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd97219_5c5b_4417_943b_072cca4466ae.slice/crio-55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd WatchSource:0}: Error finding container 55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd: Status 404 returned error can't find the container with id 55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd
Oct 29 23:29:58.622183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.622155    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4" event={"ID":"8bd97219-5c5b-4417-943b-072cca4466ae","Type":"ContainerStarted","Data":"55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd"}
Oct 29 23:29:58.762086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.762061    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:58.854047 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.854023    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\") pod \"1e555453-98e5-4aa4-97da-2a647289beba\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") "
Oct 29 23:29:58.854181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.854098    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\" (UniqueName: \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\") pod \"1e555453-98e5-4aa4-97da-2a647289beba\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") "
Oct 29 23:29:58.854227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.854212    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\") pod \"1e555453-98e5-4aa4-97da-2a647289beba\" (UID: \"1e555453-98e5-4aa4-97da-2a647289beba\") "
Oct 29 23:29:58.854423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.854398    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca" pod "1e555453-98e5-4aa4-97da-2a647289beba" (UID: "1e555453-98e5-4aa4-97da-2a647289beba"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:29:58.854710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.854694    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:58.856058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.856031    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp" pod "1e555453-98e5-4aa4-97da-2a647289beba" (UID: "1e555453-98e5-4aa4-97da-2a647289beba"). InnerVolumeSpecName "kube-api-access-8d4cp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:29:58.856144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.856102    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca" pod "1e555453-98e5-4aa4-97da-2a647289beba" (UID: "1e555453-98e5-4aa4-97da-2a647289beba"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:29:58.956181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.956134    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d4cp\" (UniqueName: \"kubernetes.io/projected/1e555453-98e5-4aa4-97da-2a647289beba-kube-api-access-8d4cp\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:58.956181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:58.956154    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1e555453-98e5-4aa4-97da-2a647289beba-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:29:59.592223 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.592192    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:29:59.631627 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.631589    5111 generic.go:356] "Generic (PLEG): container finished" podID="8bd97219-5c5b-4417-943b-072cca4466ae" containerID="6c5d0cb3f928ce31785ffc121364703f2167733740ea8ddea0b8444c958699b2" exitCode=0
Oct 29 23:29:59.632381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.631695    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4" event={"ID":"8bd97219-5c5b-4417-943b-072cca4466ae","Type":"ContainerDied","Data":"6c5d0cb3f928ce31785ffc121364703f2167733740ea8ddea0b8444c958699b2"}
Oct 29 23:29:59.633888 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.633865    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"
Oct 29 23:29:59.634043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.633882    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm" event={"ID":"1e555453-98e5-4aa4-97da-2a647289beba","Type":"ContainerDied","Data":"158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2"}
Oct 29 23:29:59.634043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:29:59.633928    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="158878de75032f2cd33014ebd8c27e070a58f96f8ed23e51e8c8f410ac16e5b2"
Oct 29 23:30:00.092543 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.092485    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:00.168191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.168162    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:00.168855 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.168836    5111 cpu_manager.go:414] "containerMap: removing container" podUID="1e555453-98e5-4aa4-97da-2a647289beba" containerName="revoke-keys"
Oct 29 23:30:00.168855 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.168855    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e555453-98e5-4aa4-97da-2a647289beba" containerName="revoke-keys"
Oct 29 23:30:00.169004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.168992    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e555453-98e5-4aa4-97da-2a647289beba" containerName="revoke-keys"
Oct 29 23:30:00.172416 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.172400    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:00.172522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.172506    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.176297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.176280    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.194302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.194280    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:00.200134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.200064    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:00.200314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.200167    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.219698 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.217663    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:00.230087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.230066    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:00.269518 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.269489    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pblq2\" (UniqueName: \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.269632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.269546    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.269678 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.269633    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh67s\" (UniqueName: \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.269831 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.269811    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gj8qf\" (UniqueName: \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.269896 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.269868    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.270045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.270004    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.371302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371246    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.371302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371298    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pblq2\" (UniqueName: \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.371462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371325    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.371462 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371356    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh67s\" (UniqueName: \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.371569 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371473    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gj8qf\" (UniqueName: \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.371569 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371505    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.371911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371891    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.372005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.371966    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.372167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.372151    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.392593 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.392563    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh67s\" (UniqueName: \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\") pod \"maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.393219 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.393194    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gj8qf\" (UniqueName: \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\") pod \"maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.393219 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.393212    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pblq2\" (UniqueName: \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\") pod \"maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.488059 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.488034    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:00.496967 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.496949    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:00.539994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.539180    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:00.786505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.786481    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:30:00.805638 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:00.805608    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda553c687_39a1_4627_b013_bb58c0ce31da.slice/crio-835175c6a26ad9b32183f89a3b394e91737cf130e69ca8a8f952233dc3d70756 WatchSource:0}: Error finding container 835175c6a26ad9b32183f89a3b394e91737cf130e69ca8a8f952233dc3d70756: Status 404 returned error can't find the container with id 835175c6a26ad9b32183f89a3b394e91737cf130e69ca8a8f952233dc3d70756
Oct 29 23:30:00.806037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.805994    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:00.820502 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:00.820476    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f70d5bc_9a45_482d_8c45_9ccc18f06646.slice/crio-a319a91094f49d71f6e1b4b24b8a593351c1a24619ca05266cafcd4f177ca52d WatchSource:0}: Error finding container a319a91094f49d71f6e1b4b24b8a593351c1a24619ca05266cafcd4f177ca52d: Status 404 returned error can't find the container with id a319a91094f49d71f6e1b4b24b8a593351c1a24619ca05266cafcd4f177ca52d
Oct 29 23:30:00.820921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.820898    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:00.881041 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.880994    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\") pod \"8bd97219-5c5b-4417-943b-072cca4466ae\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") "
Oct 29 23:30:00.881146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.881129    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\") pod \"8bd97219-5c5b-4417-943b-072cca4466ae\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") "
Oct 29 23:30:00.881257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.881243    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\" (UniqueName: \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\") pod \"8bd97219-5c5b-4417-943b-072cca4466ae\" (UID: \"8bd97219-5c5b-4417-943b-072cca4466ae\") "
Oct 29 23:30:00.881324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.881277    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca" pod "8bd97219-5c5b-4417-943b-072cca4466ae" (UID: "8bd97219-5c5b-4417-943b-072cca4466ae"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:00.881810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.881768    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:00.888181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.888160    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:00.972572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.972542    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv" pod "8bd97219-5c5b-4417-943b-072cca4466ae" (UID: "8bd97219-5c5b-4417-943b-072cca4466ae"). InnerVolumeSpecName "kube-api-access-pnwsv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:00.972710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.972602    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca" pod "8bd97219-5c5b-4417-943b-072cca4466ae" (UID: "8bd97219-5c5b-4417-943b-072cca4466ae"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:00.983424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.983403    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8bd97219-5c5b-4417-943b-072cca4466ae-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:00.983539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:00.983425    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnwsv\" (UniqueName: \"kubernetes.io/projected/8bd97219-5c5b-4417-943b-072cca4466ae-kube-api-access-pnwsv\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:00.987439 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:00.987412    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034fe130_c336_433c_ab5b_4f40a0cd9fb5.slice/crio-e5dc244293f037996cc8440e97040eff1c133fea817196a2528ceaf2dd995213 WatchSource:0}: Error finding container e5dc244293f037996cc8440e97040eff1c133fea817196a2528ceaf2dd995213: Status 404 returned error can't find the container with id e5dc244293f037996cc8440e97040eff1c133fea817196a2528ceaf2dd995213
Oct 29 23:30:01.074491 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.074458    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:01.652472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.652438    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerStarted","Data":"ac46e48eb5d7ab3237c830f0303c4e92a601e125a2c0f6cedac23a1de6850629"}
Oct 29 23:30:01.652628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.652501    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerStarted","Data":"a319a91094f49d71f6e1b4b24b8a593351c1a24619ca05266cafcd4f177ca52d"}
Oct 29 23:30:01.654146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.654114    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerStarted","Data":"b6fb0eb69ac6a145feb5549e75cc23af361bb5da69d35e7516d81e763e67c893"}
Oct 29 23:30:01.654280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.654173    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerStarted","Data":"835175c6a26ad9b32183f89a3b394e91737cf130e69ca8a8f952233dc3d70756"}
Oct 29 23:30:01.655445 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.655420    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4" event={"ID":"8bd97219-5c5b-4417-943b-072cca4466ae","Type":"ContainerDied","Data":"55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd"}
Oct 29 23:30:01.655530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.655465    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55cfaad98ff4a1361ebff1154ca7fb8c3963d95ac98f8eda7ad9a59e012844fd"
Oct 29 23:30:01.655530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.655431    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"
Oct 29 23:30:01.657083 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.657053    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerStarted","Data":"a40830b8876b00842ffa092c8fbabb4079bae19a0383782dc34c99d47d036949"}
Oct 29 23:30:01.657179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.657095    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerStarted","Data":"e5dc244293f037996cc8440e97040eff1c133fea817196a2528ceaf2dd995213"}
Oct 29 23:30:01.696683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.696643    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" podStartSLOduration=1.696630269 podStartE2EDuration="1.696630269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:01.693495724 +0000 UTC m=+1012.779778860" watchObservedRunningTime="2026-10-29 23:30:01.696630269 +0000 UTC m=+1012.782913401"
Oct 29 23:30:01.737833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.737795    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" podStartSLOduration=1.737784719 podStartE2EDuration="1.737784719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:01.735078955 +0000 UTC m=+1012.821362089" watchObservedRunningTime="2026-10-29 23:30:01.737784719 +0000 UTC m=+1012.824067853"
Oct 29 23:30:01.793134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:01.793095    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" podStartSLOduration=1.793082147 podStartE2EDuration="1.793082147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:01.791180553 +0000 UTC m=+1012.877463686" watchObservedRunningTime="2026-10-29 23:30:01.793082147 +0000 UTC m=+1012.879365280"
Oct 29 23:30:02.581476 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.581432    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:02.666407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.666372    5111 generic.go:356] "Generic (PLEG): container finished" podID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerID="ac46e48eb5d7ab3237c830f0303c4e92a601e125a2c0f6cedac23a1de6850629" exitCode=60
Oct 29 23:30:02.666618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.666447    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerDied","Data":"ac46e48eb5d7ab3237c830f0303c4e92a601e125a2c0f6cedac23a1de6850629"}
Oct 29 23:30:02.666770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.666754    5111 scope.go:122] "RemoveContainer" containerID="ac46e48eb5d7ab3237c830f0303c4e92a601e125a2c0f6cedac23a1de6850629"
Oct 29 23:30:02.668283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.668260    5111 generic.go:356] "Generic (PLEG): container finished" podID="a553c687-39a1-4627-b013-bb58c0ce31da" containerID="b6fb0eb69ac6a145feb5549e75cc23af361bb5da69d35e7516d81e763e67c893" exitCode=60
Oct 29 23:30:02.668394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.668341    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerDied","Data":"b6fb0eb69ac6a145feb5549e75cc23af361bb5da69d35e7516d81e763e67c893"}
Oct 29 23:30:02.668552 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.668534    5111 scope.go:122] "RemoveContainer" containerID="b6fb0eb69ac6a145feb5549e75cc23af361bb5da69d35e7516d81e763e67c893"
Oct 29 23:30:02.670226 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.670164    5111 generic.go:356] "Generic (PLEG): container finished" podID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerID="a40830b8876b00842ffa092c8fbabb4079bae19a0383782dc34c99d47d036949" exitCode=60
Oct 29 23:30:02.670329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.670299    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerDied","Data":"a40830b8876b00842ffa092c8fbabb4079bae19a0383782dc34c99d47d036949"}
Oct 29 23:30:02.670639 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:02.670619    5111 scope.go:122] "RemoveContainer" containerID="a40830b8876b00842ffa092c8fbabb4079bae19a0383782dc34c99d47d036949"
Oct 29 23:30:03.702547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.696230    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerStarted","Data":"ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12"}
Oct 29 23:30:03.702547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.699814    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerStarted","Data":"b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"}
Oct 29 23:30:03.703049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.702808    5111 generic.go:356] "Generic (PLEG): container finished" podID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerID="c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac" exitCode=60
Oct 29 23:30:03.703049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.702893    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerDied","Data":"c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac"}
Oct 29 23:30:03.703049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.702925    5111 scope.go:122] "RemoveContainer" containerID="a40830b8876b00842ffa092c8fbabb4079bae19a0383782dc34c99d47d036949"
Oct 29 23:30:03.703315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.703298    5111 scope.go:122] "RemoveContainer" containerID="c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac"
Oct 29 23:30:03.703579 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:03.703558    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2_odh-ai-gateway-infra(034fe130-c336-433c-ab5b-4f40a0cd9fb5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5"
Oct 29 23:30:03.728065 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.728042    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:03.866957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:03.866923    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:04.035475 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.035442    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:04.714305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.714272    5111 generic.go:356] "Generic (PLEG): container finished" podID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerID="ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12" exitCode=60
Oct 29 23:30:04.714732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.714357    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerDied","Data":"ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12"}
Oct 29 23:30:04.714732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.714423    5111 scope.go:122] "RemoveContainer" containerID="ac46e48eb5d7ab3237c830f0303c4e92a601e125a2c0f6cedac23a1de6850629"
Oct 29 23:30:04.714732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.714616    5111 scope.go:122] "RemoveContainer" containerID="ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12"
Oct 29 23:30:04.715023 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:04.714983    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj_odh-ai-gateway-infra(8f70d5bc-9a45-482d-8c45-9ccc18f06646)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646"
Oct 29 23:30:04.717207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.717145    5111 generic.go:356] "Generic (PLEG): container finished" podID="a553c687-39a1-4627-b013-bb58c0ce31da" containerID="b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0" exitCode=60
Oct 29 23:30:04.717305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.717225    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerDied","Data":"b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"}
Oct 29 23:30:04.717490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.717474    5111 scope.go:122] "RemoveContainer" containerID="b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"
Oct 29 23:30:04.717788 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:04.717762    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6_odh-ai-gateway-infra(a553c687-39a1-4627-b013-bb58c0ce31da)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" podUID="a553c687-39a1-4627-b013-bb58c0ce31da"
Oct 29 23:30:04.719314 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.719293    5111 scope.go:122] "RemoveContainer" containerID="c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac"
Oct 29 23:30:04.719622 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:04.719596    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2_odh-ai-gateway-infra(034fe130-c336-433c-ab5b-4f40a0cd9fb5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5"
Oct 29 23:30:04.728953 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:04.728933    5111 scope.go:122] "RemoveContainer" containerID="b6fb0eb69ac6a145feb5549e75cc23af361bb5da69d35e7516d81e763e67c893"
Oct 29 23:30:05.164343 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:05.164308    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:05.190479 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:05.190439    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:05.747770 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:05.747738    5111 scope.go:122] "RemoveContainer" containerID="ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12"
Oct 29 23:30:05.749267 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:05.749223    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj_odh-ai-gateway-infra(8f70d5bc-9a45-482d-8c45-9ccc18f06646)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646"
Oct 29 23:30:05.750580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:05.750555    5111 scope.go:122] "RemoveContainer" containerID="b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"
Oct 29 23:30:05.750933 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:05.750911    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6_odh-ai-gateway-infra(a553c687-39a1-4627-b013-bb58c0ce31da)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" podUID="a553c687-39a1-4627-b013-bb58c0ce31da"
Oct 29 23:30:05.846851 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:05.846819    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:06.002596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.002523    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:06.160020 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.159970    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:06.512400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.512363    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:06.910405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.910381    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:06.969475 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.969446    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\") pod \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") "
Oct 29 23:30:06.969624 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.969509    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\" (UniqueName: \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\") pod \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\" (UID: \"8f70d5bc-9a45-482d-8c45-9ccc18f06646\") "
Oct 29 23:30:06.969783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.969759    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca" pod "8f70d5bc-9a45-482d-8c45-9ccc18f06646" (UID: "8f70d5bc-9a45-482d-8c45-9ccc18f06646"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:06.970151 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.970120    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f70d5bc-9a45-482d-8c45-9ccc18f06646-maas-api-service-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:06.971721 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:06.971692    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2" pod "8f70d5bc-9a45-482d-8c45-9ccc18f06646" (UID: "8f70d5bc-9a45-482d-8c45-9ccc18f06646"). InnerVolumeSpecName "kube-api-access-pblq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:07.071273 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.071245    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pblq2\" (UniqueName: \"kubernetes.io/projected/8f70d5bc-9a45-482d-8c45-9ccc18f06646-kube-api-access-pblq2\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:07.363469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.363436    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:07.446653 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.446626    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"]
Oct 29 23:30:07.447182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447169    5111 cpu_manager.go:414] "containerMap: removing container" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447183    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447219    5111 cpu_manager.go:414] "containerMap: removing container" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447225    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447233    5111 cpu_manager.go:414] "containerMap: removing container" podUID="8bd97219-5c5b-4417-943b-072cca4466ae" containerName="revoke-keys"
Oct 29 23:30:07.447243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447237    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8bd97219-5c5b-4417-943b-072cca4466ae" containerName="revoke-keys"
Oct 29 23:30:07.447386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447319    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447328    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" containerName="cleanup"
Oct 29 23:30:07.447386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.447334    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="8bd97219-5c5b-4417-943b-072cca4466ae" containerName="revoke-keys"
Oct 29 23:30:07.450745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.450731    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.470587    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"]
Oct 29 23:30:07.480064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.480045    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:30:07.522633 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.522602    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:07.577080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.577042    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.577342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.577317    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.577410 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.577364    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d6sz\" (UniqueName: \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.678836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.678761    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.678836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.678805    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d6sz\" (UniqueName: \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.679044 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.678867    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.679534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.679505    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.681166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.681141    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.700064 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.700040    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d6sz\" (UniqueName: \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\") pod \"maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.764629 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.764600    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:07.766166 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.766144    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"
Oct 29 23:30:07.766289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.766146    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj" event={"ID":"8f70d5bc-9a45-482d-8c45-9ccc18f06646","Type":"ContainerDied","Data":"a319a91094f49d71f6e1b4b24b8a593351c1a24619ca05266cafcd4f177ca52d"}
Oct 29 23:30:07.766349 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.766300    5111 scope.go:122] "RemoveContainer" containerID="ea97edaceb46d558d3d65b97f034b144ce900cbe9b3c1a89a58511645f6d1d12"
Oct 29 23:30:07.825893 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.825860    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:07.834453 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:07.834429    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-313d9c-29888610-l5pmj"]
Oct 29 23:30:08.082300 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:08.082273    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a5bb98_9467_40b4_91ef_7adc9e570647.slice/crio-139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b WatchSource:0}: Error finding container 139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b: Status 404 returned error can't find the container with id 139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b
Oct 29 23:30:08.083081 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.082699    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"]
Oct 29 23:30:08.142875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.142851    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"]
Oct 29 23:30:08.149311 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.149291    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.169387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.169357    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"]
Oct 29 23:30:08.214333 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.214307    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f46r\" (UniqueName: \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.214477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.214431    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.214477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.214469    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.315679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.315651    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f46r\" (UniqueName: \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.315830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.315776    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.315830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.315815    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.316441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.316421    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.318417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.318396    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.336802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.336747    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f46r\" (UniqueName: \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\") pod \"maas-api-revoke-sub-f60470af-5rc4k\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.469033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.469010    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:08.524668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.524643    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:08.641176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.641117    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"]
Oct 29 23:30:08.645545 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.645530    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.663845 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.663476    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"]
Oct 29 23:30:08.721043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.721010    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.721152 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.721048    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sncv\" (UniqueName: \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.721202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.721164    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.775442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.774734    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"]
Oct 29 23:30:08.776105 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:08.776081    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d245a8_744e_4a7e_aaee_fb000959f749.slice/crio-fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f WatchSource:0}: Error finding container fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f: Status 404 returned error can't find the container with id fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f
Oct 29 23:30:08.784724 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.784695    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s" event={"ID":"60a5bb98-9467-40b4-91ef-7adc9e570647","Type":"ContainerStarted","Data":"d820e5ae25c3523f3aba125a03df6a32384e8777bcf916370751063a41cb5033"}
Oct 29 23:30:08.785036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.784749    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s" event={"ID":"60a5bb98-9467-40b4-91ef-7adc9e570647","Type":"ContainerStarted","Data":"139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b"}
Oct 29 23:30:08.822186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.822163    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.822278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.822194    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sncv\" (UniqueName: \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.822278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.822231    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.822705 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.822683    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.824374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.824353    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.828376 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.828341    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s" podStartSLOduration=1.8283301299999999 podStartE2EDuration="1.82833013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:08.825585302 +0000 UTC m=+1019.911868430" watchObservedRunningTime="2026-10-29 23:30:08.82833013 +0000 UTC m=+1019.914613262"
Oct 29 23:30:08.843257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.843238    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sncv\" (UniqueName: \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\") pod \"maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:08.962125 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:08.962063    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:09.075629 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.075598    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f70d5bc-9a45-482d-8c45-9ccc18f06646" path="/var/lib/kubelet/pods/8f70d5bc-9a45-482d-8c45-9ccc18f06646/volumes"
Oct 29 23:30:09.327809 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.327763    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"]
Oct 29 23:30:09.408130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.408091    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:09.571239 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.571087    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:09.720701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.720664    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:09.795161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.795128    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2" event={"ID":"65361eaa-48a4-442a-92d1-a090fefbb3f5","Type":"ContainerStarted","Data":"c052fdab00780833d8a4eb49ed4abbc3fd0f87a17ed9063e9ed7336169e86f70"}
Oct 29 23:30:09.795328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.795188    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2" event={"ID":"65361eaa-48a4-442a-92d1-a090fefbb3f5","Type":"ContainerStarted","Data":"3e337fffd6c0f904b70c27278c14d5ff6e0ea9ccf6fe66ca4dbc8e05dae6d3a2"}
Oct 29 23:30:09.797488 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.797455    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k" event={"ID":"32d245a8-744e-4a7e-aaee-fb000959f749","Type":"ContainerStarted","Data":"64ece6ae73c83dc8a5aa4487f91a1f328f1b77eb711f430e4b8c1f45fd5b10f3"}
Oct 29 23:30:09.797610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.797520    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k" event={"ID":"32d245a8-744e-4a7e-aaee-fb000959f749","Type":"ContainerStarted","Data":"fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f"}
Oct 29 23:30:09.799777 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.799749    5111 generic.go:356] "Generic (PLEG): container finished" podID="60a5bb98-9467-40b4-91ef-7adc9e570647" containerID="d820e5ae25c3523f3aba125a03df6a32384e8777bcf916370751063a41cb5033" exitCode=0
Oct 29 23:30:09.799863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.799781    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s" event={"ID":"60a5bb98-9467-40b4-91ef-7adc9e570647","Type":"ContainerDied","Data":"d820e5ae25c3523f3aba125a03df6a32384e8777bcf916370751063a41cb5033"}
Oct 29 23:30:09.839943 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.839901    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2" podStartSLOduration=1.839889208 podStartE2EDuration="1.839889208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:09.837454819 +0000 UTC m=+1020.923737952" watchObservedRunningTime="2026-10-29 23:30:09.839889208 +0000 UTC m=+1020.926172340"
Oct 29 23:30:09.879097 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:09.879026    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k" podStartSLOduration=1.879008996 podStartE2EDuration="1.879008996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:09.877788302 +0000 UTC m=+1020.964071437" watchObservedRunningTime="2026-10-29 23:30:09.879008996 +0000 UTC m=+1020.965292145"
Oct 29 23:30:10.462408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.462375    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:10.807661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.807625    5111 generic.go:356] "Generic (PLEG): container finished" podID="65361eaa-48a4-442a-92d1-a090fefbb3f5" containerID="c052fdab00780833d8a4eb49ed4abbc3fd0f87a17ed9063e9ed7336169e86f70" exitCode=0
Oct 29 23:30:10.807818 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.807709    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2" event={"ID":"65361eaa-48a4-442a-92d1-a090fefbb3f5","Type":"ContainerDied","Data":"c052fdab00780833d8a4eb49ed4abbc3fd0f87a17ed9063e9ed7336169e86f70"}
Oct 29 23:30:10.809310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.809290    5111 generic.go:356] "Generic (PLEG): container finished" podID="32d245a8-744e-4a7e-aaee-fb000959f749" containerID="64ece6ae73c83dc8a5aa4487f91a1f328f1b77eb711f430e4b8c1f45fd5b10f3" exitCode=0
Oct 29 23:30:10.809404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.809371    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k" event={"ID":"32d245a8-744e-4a7e-aaee-fb000959f749","Type":"ContainerDied","Data":"64ece6ae73c83dc8a5aa4487f91a1f328f1b77eb711f430e4b8c1f45fd5b10f3"}
Oct 29 23:30:10.993611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:10.993579    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:11.091087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.091050    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:11.161574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.161541    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\" (UniqueName: \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\") pod \"60a5bb98-9467-40b4-91ef-7adc9e570647\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") "
Oct 29 23:30:11.161791 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.161597    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\") pod \"60a5bb98-9467-40b4-91ef-7adc9e570647\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") "
Oct 29 23:30:11.161791 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.161661    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\") pod \"60a5bb98-9467-40b4-91ef-7adc9e570647\" (UID: \"60a5bb98-9467-40b4-91ef-7adc9e570647\") "
Oct 29 23:30:11.162352 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.162322    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca" pod "60a5bb98-9467-40b4-91ef-7adc9e570647" (UID: "60a5bb98-9467-40b4-91ef-7adc9e570647"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:11.164057 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.164030    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca" pod "60a5bb98-9467-40b4-91ef-7adc9e570647" (UID: "60a5bb98-9467-40b4-91ef-7adc9e570647"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:11.164457 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.164437    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz" pod "60a5bb98-9467-40b4-91ef-7adc9e570647" (UID: "60a5bb98-9467-40b4-91ef-7adc9e570647"). InnerVolumeSpecName "kube-api-access-4d6sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:11.264325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.264300    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d6sz\" (UniqueName: \"kubernetes.io/projected/60a5bb98-9467-40b4-91ef-7adc9e570647-kube-api-access-4d6sz\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:11.264325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.264326    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:11.264501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.264342    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/60a5bb98-9467-40b4-91ef-7adc9e570647-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:11.688856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.688799    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main" containerID="cri-o://91b3dcd768420f1fd79c92a282a8b087bf5fdb9bba4c9f76bbefa0bfee55677c" gracePeriod=45
Oct 29 23:30:11.689285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.688799    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main" containerID="cri-o://b9d8529576d89463940ec066f9d0d3ac4c7bffb5c33e9cb142a82a78761f2b74" gracePeriod=45
Oct 29 23:30:11.817206 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.817177    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"
Oct 29 23:30:11.817206 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.817189    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s" event={"ID":"60a5bb98-9467-40b4-91ef-7adc9e570647","Type":"ContainerDied","Data":"139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b"}
Oct 29 23:30:11.817415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.817243    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="139dd09bf3182907df35ccbb1f91b76d5c932a934467a0ec10432344887b461b"
Oct 29 23:30:11.822253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.822226    5111 generic.go:356] "Generic (PLEG): container finished" podID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerID="91b3dcd768420f1fd79c92a282a8b087bf5fdb9bba4c9f76bbefa0bfee55677c" exitCode=0
Oct 29 23:30:11.822369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.822271    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" event={"ID":"de3882a9-9c6c-4f3d-9a81-fac7a96140e3","Type":"ContainerDied","Data":"91b3dcd768420f1fd79c92a282a8b087bf5fdb9bba4c9f76bbefa0bfee55677c"}
Oct 29 23:30:11.824412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.824262    5111 generic.go:356] "Generic (PLEG): container finished" podID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerID="b9d8529576d89463940ec066f9d0d3ac4c7bffb5c33e9cb142a82a78761f2b74" exitCode=0
Oct 29 23:30:11.824412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:11.824340    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" event={"ID":"aa1774dc-4bdb-443d-a0a6-be41b652a970","Type":"ContainerDied","Data":"b9d8529576d89463940ec066f9d0d3ac4c7bffb5c33e9cb142a82a78761f2b74"}
Oct 29 23:30:12.025670 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.025471    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:30:12.081353 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.081328    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:30:12.085199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.085175    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:12.089807 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.089786    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:12.183009 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.182969    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.183167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183033    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.183167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183099    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.183167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183127    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\") pod \"32d245a8-744e-4a7e-aaee-fb000959f749\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") "
Oct 29 23:30:12.183167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183162    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\" (UniqueName: \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\") pod \"65361eaa-48a4-442a-92d1-a090fefbb3f5\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") "
Oct 29 23:30:12.183388 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183190    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.183562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183538    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\") pod \"65361eaa-48a4-442a-92d1-a090fefbb3f5\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") "
Oct 29 23:30:12.183635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183568    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.183688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183631    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.183688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183638    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.183688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183660    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\") pod \"32d245a8-744e-4a7e-aaee-fb000959f749\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") "
Oct 29 23:30:12.183844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183689    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\") pod \"65361eaa-48a4-442a-92d1-a090fefbb3f5\" (UID: \"65361eaa-48a4-442a-92d1-a090fefbb3f5\") "
Oct 29 23:30:12.183844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183767    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\" (UniqueName: \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.183844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183832    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.184001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183869    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.184001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183912    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\" (UniqueName: \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.184001 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183952    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca" pod "65361eaa-48a4-442a-92d1-a090fefbb3f5" (UID: "65361eaa-48a4-442a-92d1-a090fefbb3f5"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:12.184158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184002    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\" (UniqueName: \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\") pod \"32d245a8-744e-4a7e-aaee-fb000959f749\" (UID: \"32d245a8-744e-4a7e-aaee-fb000959f749\") "
Oct 29 23:30:12.184158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.183968    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.184158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184053    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\") pod \"aa1774dc-4bdb-443d-a0a6-be41b652a970\" (UID: \"aa1774dc-4bdb-443d-a0a6-be41b652a970\") "
Oct 29 23:30:12.184158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184130    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\" (UniqueName: \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.184338 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184159    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\") pod \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\" (UID: \"de3882a9-9c6c-4f3d-9a81-fac7a96140e3\") "
Oct 29 23:30:12.184338 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184196    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184703    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184763    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca" pod "32d245a8-744e-4a7e-aaee-fb000959f749" (UID: "32d245a8-744e-4a7e-aaee-fb000959f749"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184913    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184931    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184941    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184949    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184957    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184955    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.185055 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.184966    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.189957 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.189920    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.190158 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.190114    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:12.190258 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.190194    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:12.190329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.190311    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv" pod "65361eaa-48a4-442a-92d1-a090fefbb3f5" (UID: "65361eaa-48a4-442a-92d1-a090fefbb3f5"). InnerVolumeSpecName "kube-api-access-4sncv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:12.190393 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.190366    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca" pod "32d245a8-744e-4a7e-aaee-fb000959f749" (UID: "32d245a8-744e-4a7e-aaee-fb000959f749"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:12.190451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.190414    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca" pod "65361eaa-48a4-442a-92d1-a090fefbb3f5" (UID: "65361eaa-48a4-442a-92d1-a090fefbb3f5"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:12.191740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.191038    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:12.191740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.191519    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv" pod "aa1774dc-4bdb-443d-a0a6-be41b652a970" (UID: "aa1774dc-4bdb-443d-a0a6-be41b652a970"). InnerVolumeSpecName "kube-api-access-lknwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:12.191740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.191633    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r" pod "32d245a8-744e-4a7e-aaee-fb000959f749" (UID: "32d245a8-744e-4a7e-aaee-fb000959f749"). InnerVolumeSpecName "kube-api-access-7f46r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:12.192411 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.192390    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g" pod "de3882a9-9c6c-4f3d-9a81-fac7a96140e3" (UID: "de3882a9-9c6c-4f3d-9a81-fac7a96140e3"). InnerVolumeSpecName "kube-api-access-psq6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:12.200161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.200136    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"]
Oct 29 23:30:12.201033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201012    5111 cpu_manager.go:414] "containerMap: removing container" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main"
Oct 29 23:30:12.201033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201034    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main"
Oct 29 23:30:12.201160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201069    5111 cpu_manager.go:414] "containerMap: removing container" podUID="65361eaa-48a4-442a-92d1-a090fefbb3f5" containerName="revoke-keys"
Oct 29 23:30:12.201160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201078    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="65361eaa-48a4-442a-92d1-a090fefbb3f5" containerName="revoke-keys"
Oct 29 23:30:12.201160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201119    5111 cpu_manager.go:414] "containerMap: removing container" podUID="32d245a8-744e-4a7e-aaee-fb000959f749" containerName="revoke-keys"
Oct 29 23:30:12.201160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201129    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32d245a8-744e-4a7e-aaee-fb000959f749" containerName="revoke-keys"
Oct 29 23:30:12.201160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201153    5111 cpu_manager.go:414] "containerMap: removing container" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main"
Oct 29 23:30:12.201302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201161    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main"
Oct 29 23:30:12.201302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201181    5111 cpu_manager.go:414] "containerMap: removing container" podUID="60a5bb98-9467-40b4-91ef-7adc9e570647" containerName="revoke-keys"
Oct 29 23:30:12.201302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201189    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60a5bb98-9467-40b4-91ef-7adc9e570647" containerName="revoke-keys"
Oct 29 23:30:12.201387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201344    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="32d245a8-744e-4a7e-aaee-fb000959f749" containerName="revoke-keys"
Oct 29 23:30:12.201387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201360    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="65361eaa-48a4-442a-92d1-a090fefbb3f5" containerName="revoke-keys"
Oct 29 23:30:12.201387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201371    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" containerName="main"
Oct 29 23:30:12.201387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201385    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="60a5bb98-9467-40b4-91ef-7adc9e570647" containerName="revoke-keys"
Oct 29 23:30:12.201501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.201397    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" containerName="main"
Oct 29 23:30:12.207601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.207579    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.221433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.221409    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"]
Oct 29 23:30:12.240444 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.238921    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"]
Oct 29 23:30:12.240444 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.239132    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.246649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.246616    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:30:12.256198 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.255212    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"]
Oct 29 23:30:12.263262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.263231    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:30:12.266823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.266800    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286127    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f46r\" (UniqueName: \"kubernetes.io/projected/32d245a8-744e-4a7e-aaee-fb000959f749-kube-api-access-7f46r\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286155    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286171    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286184    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aa1774dc-4bdb-443d-a0a6-be41b652a970-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286196    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286209    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4sncv\" (UniqueName: \"kubernetes.io/projected/65361eaa-48a4-442a-92d1-a090fefbb3f5-kube-api-access-4sncv\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286220    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aa1774dc-4bdb-443d-a0a6-be41b652a970-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286234    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/32d245a8-744e-4a7e-aaee-fb000959f749-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286248    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/65361eaa-48a4-442a-92d1-a090fefbb3f5-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286262    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psq6g\" (UniqueName: \"kubernetes.io/projected/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-kube-api-access-psq6g\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286274    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de3882a9-9c6c-4f3d-9a81-fac7a96140e3-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.287116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.286286    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lknwv\" (UniqueName: \"kubernetes.io/projected/aa1774dc-4bdb-443d-a0a6-be41b652a970-kube-api-access-lknwv\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:12.387272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387240    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f987b\" (UniqueName: \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.387440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387281    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.387440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387387    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.387517 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387506    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.387583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387571    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.387680 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.387662    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68dtj\" (UniqueName: \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.423848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.423823    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:12.488574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488541    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.488695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488591    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68dtj\" (UniqueName: \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.488695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488640    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f987b\" (UniqueName: \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.488695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488667    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.488886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488712    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.488886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.488847    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.489509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.489482    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.489626 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.489559    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.491400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.491355    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.491400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.491356    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.508412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.508392    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f987b\" (UniqueName: \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\") pod \"maas-api-revoke-sub-e9e2de7d-znmld\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.508491 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.508473    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68dtj\" (UniqueName: \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\") pod \"maas-api-revoke-sub-bdb9f8fd-l9tmf\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.526468 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.526449    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:12.560696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.560674    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:12.722837 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.722800    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:12.844139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.843930    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947" event={"ID":"de3882a9-9c6c-4f3d-9a81-fac7a96140e3","Type":"ContainerDied","Data":"b6b0e8e10dae8546c716f2fcdb031d3d6ba955d0512b0ae2495cd7ba6046a900"}
Oct 29 23:30:12.844409 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.844380    5111 scope.go:122] "RemoveContainer" containerID="91b3dcd768420f1fd79c92a282a8b087bf5fdb9bba4c9f76bbefa0bfee55677c"
Oct 29 23:30:12.844726 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.844707    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"
Oct 29 23:30:12.850824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.848922    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2" event={"ID":"65361eaa-48a4-442a-92d1-a090fefbb3f5","Type":"ContainerDied","Data":"3e337fffd6c0f904b70c27278c14d5ff6e0ea9ccf6fe66ca4dbc8e05dae6d3a2"}
Oct 29 23:30:12.850824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.848968    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e337fffd6c0f904b70c27278c14d5ff6e0ea9ccf6fe66ca4dbc8e05dae6d3a2"
Oct 29 23:30:12.850824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.849059    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"
Oct 29 23:30:12.860347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.855770    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"
Oct 29 23:30:12.860347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.855787    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k" event={"ID":"32d245a8-744e-4a7e-aaee-fb000959f749","Type":"ContainerDied","Data":"fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f"}
Oct 29 23:30:12.860347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.855828    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb56b254d26eaf3fef26282603da2eda2b31ef2cebf00e6a3db7aa12298ea74f"
Oct 29 23:30:12.860347 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.859942    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"]
Oct 29 23:30:12.863984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.863669    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"
Oct 29 23:30:12.863984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.863722    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz" event={"ID":"aa1774dc-4bdb-443d-a0a6-be41b652a970","Type":"ContainerDied","Data":"2e57d9e02f3072c858042af14cf322188de16c165ca081ea78d4c055b54b5f37"}
Oct 29 23:30:12.863984 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.863757    5111 scope.go:122] "RemoveContainer" containerID="b9d8529576d89463940ec066f9d0d3ac4c7bffb5c33e9cb142a82a78761f2b74"
Oct 29 23:30:12.865128 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:12.865104    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87007b6_23f4_402a_9459_62dfb6254f7e.slice/crio-3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e WatchSource:0}: Error finding container 3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e: Status 404 returned error can't find the container with id 3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e
Oct 29 23:30:12.892513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.892490    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"]
Oct 29 23:30:12.936343 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.936319    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:30:12.941120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.941097    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/simulator-w4-032fa8-kserve-55dfbf5775-k5dmz"]
Oct 29 23:30:12.990566 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:12.990526    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:30:13.000573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.000547    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-032fa8/distinct-w4-032fa8-kserve-dfb8f989c-6q947"]
Oct 29 23:30:13.063454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.063427    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa1774dc-4bdb-443d-a0a6-be41b652a970" path="/var/lib/kubelet/pods/aa1774dc-4bdb-443d-a0a6-be41b652a970/volumes"
Oct 29 23:30:13.063871 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.063857    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de3882a9-9c6c-4f3d-9a81-fac7a96140e3" path="/var/lib/kubelet/pods/de3882a9-9c6c-4f3d-9a81-fac7a96140e3/volumes"
Oct 29 23:30:13.210268 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.210220    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:13.400929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.400882    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:13.504592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.503929    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:30:13.504592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.504209    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" containerID="cri-o://80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e" gracePeriod=30
Oct 29 23:30:13.513489 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.513463    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:30:13.514554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.513710    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" containerID="cri-o://56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df" gracePeriod=30
Oct 29 23:30:13.519009 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.518952    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:30:13.519169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.519147    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy" containerID="cri-o://78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8" gracePeriod=30
Oct 29 23:30:13.531742 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.531715    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:30:13.598413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.598383    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:13.767790 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.767708    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:13.876850 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.876792    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf" event={"ID":"f87007b6-23f4-402a-9459-62dfb6254f7e","Type":"ContainerStarted","Data":"62a25847d484542e98cc8b123598d0a0c6e807300965a32f747bc9a719201cf7"}
Oct 29 23:30:13.877046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.876879    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf" event={"ID":"f87007b6-23f4-402a-9459-62dfb6254f7e","Type":"ContainerStarted","Data":"3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e"}
Oct 29 23:30:13.881802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.881568    5111 generic.go:356] "Generic (PLEG): container finished" podID="ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" containerID="fbc7ebe3dd8471bbbbb9e3ce4251b1deed0f914daaa6ad06978c4b8fa0925dc2" exitCode=0
Oct 29 23:30:13.881802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.881677    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld" event={"ID":"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790","Type":"ContainerDied","Data":"fbc7ebe3dd8471bbbbb9e3ce4251b1deed0f914daaa6ad06978c4b8fa0925dc2"}
Oct 29 23:30:13.881802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.881750    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld" event={"ID":"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790","Type":"ContainerStarted","Data":"be996499d1795f6f702be0929a5d1cd8d15250e46ec8703a52e6d528c792982d"}
Oct 29 23:30:13.921014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:13.920922    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf" podStartSLOduration=1.920907974 podStartE2EDuration="1.920907974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:13.918332427 +0000 UTC m=+1025.004615562" watchObservedRunningTime="2026-10-29 23:30:13.920907974 +0000 UTC m=+1025.007191107"
Oct 29 23:30:14.353156 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:14.353120    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:14.914998 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:14.914041    5111 generic.go:356] "Generic (PLEG): container finished" podID="f87007b6-23f4-402a-9459-62dfb6254f7e" containerID="62a25847d484542e98cc8b123598d0a0c6e807300965a32f747bc9a719201cf7" exitCode=0
Oct 29 23:30:14.915577 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:14.915550    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf" event={"ID":"f87007b6-23f4-402a-9459-62dfb6254f7e","Type":"ContainerDied","Data":"62a25847d484542e98cc8b123598d0a0c6e807300965a32f747bc9a719201cf7"}
Oct 29 23:30:15.169186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.169130    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:15.282687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.282655    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\" (UniqueName: \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\") pod \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") "
Oct 29 23:30:15.282860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.282777    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\") pod \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") "
Oct 29 23:30:15.282860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.282853    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\") pod \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\" (UID: \"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790\") "
Oct 29 23:30:15.283238 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.283209    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca" pod "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" (UID: "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:15.283434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.283417    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:15.284755 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.284726    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca" pod "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" (UID: "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:15.284997 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.284954    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b" pod "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" (UID: "ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790"). InnerVolumeSpecName "kube-api-access-f987b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:15.391996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.388268    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f987b\" (UniqueName: \"kubernetes.io/projected/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-kube-api-access-f987b\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:15.391996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.388314    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:15.999999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:15.991291    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"
Oct 29 23:30:16.011550 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.007924    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld" event={"ID":"ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790","Type":"ContainerDied","Data":"be996499d1795f6f702be0929a5d1cd8d15250e46ec8703a52e6d528c792982d"}
Oct 29 23:30:16.011550 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.008007    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="be996499d1795f6f702be0929a5d1cd8d15250e46ec8703a52e6d528c792982d"
Oct 29 23:30:16.067021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.057774    5111 scope.go:122] "RemoveContainer" containerID="b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"
Oct 29 23:30:16.474142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.472284    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:16.636688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.484054    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:16.636688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.546703    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": context deadline exceeded"
Oct 29 23:30:16.644996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.642053    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded"
Oct 29 23:30:16.644996 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.642382    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:16.961093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.936055    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" probeResult="failure" output="Get \"https://10.132.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:16.961093 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.952091    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:30:16.987353 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.972506    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:17.010008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:16.991172    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:17.085037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.064380    5111 scope.go:122] "RemoveContainer" containerID="c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac"
Oct 29 23:30:17.111000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.102033    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\" (UniqueName: \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\") pod \"f87007b6-23f4-402a-9459-62dfb6254f7e\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") "
Oct 29 23:30:17.111000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.102349    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\") pod \"f87007b6-23f4-402a-9459-62dfb6254f7e\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") "
Oct 29 23:30:17.111000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.102396    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\") pod \"f87007b6-23f4-402a-9459-62dfb6254f7e\" (UID: \"f87007b6-23f4-402a-9459-62dfb6254f7e\") "
Oct 29 23:30:17.128995 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.113028    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca" pod "f87007b6-23f4-402a-9459-62dfb6254f7e" (UID: "f87007b6-23f4-402a-9459-62dfb6254f7e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:17.152614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.133516    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca" pod "f87007b6-23f4-402a-9459-62dfb6254f7e" (UID: "f87007b6-23f4-402a-9459-62dfb6254f7e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:17.285058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.217916    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:17.285058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.217948    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f87007b6-23f4-402a-9459-62dfb6254f7e-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:17.285058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.248910    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj" pod "f87007b6-23f4-402a-9459-62dfb6254f7e" (UID: "f87007b6-23f4-402a-9459-62dfb6254f7e"). InnerVolumeSpecName "kube-api-access-68dtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:17.347038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.326250    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68dtj\" (UniqueName: \"kubernetes.io/projected/f87007b6-23f4-402a-9459-62dfb6254f7e-kube-api-access-68dtj\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:17.392563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.354604    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf" event={"ID":"f87007b6-23f4-402a-9459-62dfb6254f7e","Type":"ContainerDied","Data":"3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e"}
Oct 29 23:30:17.392563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.354666    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3897d70eac736ac08753fcf86f60bd599096baa63f12adb2b9646c1e7794dc9e"
Oct 29 23:30:17.392563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.354737    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"
Oct 29 23:30:17.392563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.390212    5111 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:17.476161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.454794    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:17.652506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.648153    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:17.652506 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.649578    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:30:17.661305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.661270    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:17.662139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.662084    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:30:17.751282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.750261    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:30:17.848736 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.848533    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:30:17.968263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.967886    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" probeResult="failure" output="Get \"https://10.132.0.50:8000/ready\": context deadline exceeded"
Oct 29 23:30:17.969374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.968765    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:30:17.991039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.973396    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:17.991039 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:17.983013    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:30:18.048322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.048238    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:30:18.054161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.054138    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:30:18.132389 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.132355    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:30:18.132518 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.132407    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:30:18.366619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.366573    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerStarted","Data":"228fe973ffe69250e0ed111eec9644543c4100ec1a1a2c8763160bf3a07d3c75"}
Oct 29 23:30:18.369351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.369319    5111 generic.go:356] "Generic (PLEG): container finished" podID="a553c687-39a1-4627-b013-bb58c0ce31da" containerID="88a514db5e8703f5b7f0606347c6310f1f56f8ca716205d19478ae7c862e138d" exitCode=60
Oct 29 23:30:18.369465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.369387    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerDied","Data":"88a514db5e8703f5b7f0606347c6310f1f56f8ca716205d19478ae7c862e138d"}
Oct 29 23:30:18.369465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.369446    5111 scope.go:122] "RemoveContainer" containerID="b93c962f8b6025fb576b80823e24f0b6708ce3b0088c49ced47c923c4e7b52b0"
Oct 29 23:30:18.369857 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.369841    5111 scope.go:122] "RemoveContainer" containerID="88a514db5e8703f5b7f0606347c6310f1f56f8ca716205d19478ae7c862e138d"
Oct 29 23:30:18.370202 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:18.370168    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6_odh-ai-gateway-infra(a553c687-39a1-4627-b013-bb58c0ce31da)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" podUID="a553c687-39a1-4627-b013-bb58c0ce31da"
Oct 29 23:30:18.612916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.612876    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:18.762117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.760796    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:18.842247 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.842227    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.895716    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.895772    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.895864    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.895939    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.896003    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.896102    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.896129    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.896279    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.896309    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\") pod \"0fe04ca7-5e9c-4738-9b70-59d000efe122\" (UID: \"0fe04ca7-5e9c-4738-9b70-59d000efe122\") "
Oct 29 23:30:18.897728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.897417    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.897785 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.897733    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:18.898015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.897991    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.898279 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.898258    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.898619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.898597    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.907671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.902114    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.907671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.902117    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:18.907671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.902166    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "kube-api-access-psw7g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:18.907671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.902202    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token" pod "0fe04ca7-5e9c-4738-9b70-59d000efe122" (UID: "0fe04ca7-5e9c-4738-9b70-59d000efe122"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:18.950943 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.950919    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:30:18.953660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.953375    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:30:18.997813 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.997780    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.997969 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.997818    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.997969 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.997851    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.997969 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.997900    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.997969 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.997998    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998053    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998103    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998132    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998173    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998128    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998253    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998228    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998310    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998352    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998367    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998384    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998415    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998457    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998487    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998432    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998520    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998530    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998558    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998574    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\") pod \"b62997a5-7b30-4018-92d9-1d78d2430963\" (UID: \"b62997a5-7b30-4018-92d9-1d78d2430963\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998608    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\") pod \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\" (UID: \"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5\") "
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998687    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998755    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.998960    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999174    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999196    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b62997a5-7b30-4018-92d9-1d78d2430963-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999212    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999227    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999241    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:18.999234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999254    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psw7g\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-kube-api-access-psw7g\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999267    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999281    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999294    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999307    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999319    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999333    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999345    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999357    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fe04ca7-5e9c-4738-9b70-59d000efe122-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999370    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fe04ca7-5e9c-4738-9b70-59d000efe122-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999383    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999396    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fe04ca7-5e9c-4738-9b70-59d000efe122-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999410    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.000687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:18.999422    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.001410 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001010    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:19.001410 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001156    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:19.001510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001452    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "kube-api-access-ffn2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:19.001607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001582    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:19.001658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001620    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:19.001658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001622    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:19.001720 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.001706    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898" pod "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" (UID: "0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5"). InnerVolumeSpecName "kube-api-access-76898". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:19.002438 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.002420    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token" pod "b62997a5-7b30-4018-92d9-1d78d2430963" (UID: "b62997a5-7b30-4018-92d9-1d78d2430963"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:19.101098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101071    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101420    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101443    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101452    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b62997a5-7b30-4018-92d9-1d78d2430963-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101460    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b62997a5-7b30-4018-92d9-1d78d2430963-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101469    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76898\" (UniqueName: \"kubernetes.io/projected/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-kube-api-access-76898\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101479    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffn2h\" (UniqueName: \"kubernetes.io/projected/b62997a5-7b30-4018-92d9-1d78d2430963-kube-api-access-ffn2h\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.101562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.101490    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.377574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.377539    5111 generic.go:356] "Generic (PLEG): container finished" podID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerID="78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8" exitCode=0
Oct 29 23:30:19.377722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.377590    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"
Oct 29 23:30:19.377722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.377622    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" event={"ID":"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5","Type":"ContainerDied","Data":"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"}
Oct 29 23:30:19.377722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.377681    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7" event={"ID":"0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5","Type":"ContainerDied","Data":"5aac787b18da78b6afa399615e63675767832d1424a8ba5e00e9b5bd2f76fbf1"}
Oct 29 23:30:19.377722 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.377710    5111 scope.go:122] "RemoveContainer" containerID="78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"
Oct 29 23:30:19.379278 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.379251    5111 generic.go:356] "Generic (PLEG): container finished" podID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerID="56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df" exitCode=0
Oct 29 23:30:19.379400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.379298    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"
Oct 29 23:30:19.379400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.379328    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" event={"ID":"0fe04ca7-5e9c-4738-9b70-59d000efe122","Type":"ContainerDied","Data":"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"}
Oct 29 23:30:19.379400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.379372    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq" event={"ID":"0fe04ca7-5e9c-4738-9b70-59d000efe122","Type":"ContainerDied","Data":"174c0b8bc1bb4f5a2ecad83d0b03c897d7314d3ed00a97f88c9e784649c026d8"}
Oct 29 23:30:19.383497 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.383474    5111 generic.go:356] "Generic (PLEG): container finished" podID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerID="228fe973ffe69250e0ed111eec9644543c4100ec1a1a2c8763160bf3a07d3c75" exitCode=60
Oct 29 23:30:19.383587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.383543    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerDied","Data":"228fe973ffe69250e0ed111eec9644543c4100ec1a1a2c8763160bf3a07d3c75"}
Oct 29 23:30:19.383898 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.383882    5111 scope.go:122] "RemoveContainer" containerID="228fe973ffe69250e0ed111eec9644543c4100ec1a1a2c8763160bf3a07d3c75"
Oct 29 23:30:19.384141 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:19.384122    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2_odh-ai-gateway-infra(034fe130-c336-433c-ab5b-4f40a0cd9fb5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5"
Oct 29 23:30:19.384911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.384887    5111 generic.go:356] "Generic (PLEG): container finished" podID="b62997a5-7b30-4018-92d9-1d78d2430963" containerID="80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e" exitCode=0
Oct 29 23:30:19.385031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.384920    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"
Oct 29 23:30:19.385031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.384952    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" event={"ID":"b62997a5-7b30-4018-92d9-1d78d2430963","Type":"ContainerDied","Data":"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"}
Oct 29 23:30:19.385031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.385007    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" event={"ID":"b62997a5-7b30-4018-92d9-1d78d2430963","Type":"ContainerDied","Data":"3651c47128c2f77d68c7326343287b2a3487d3bf6b1b96f09b92f0cdf68a98d9"}
Oct 29 23:30:19.389152 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.389134    5111 scope.go:122] "RemoveContainer" containerID="78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"
Oct 29 23:30:19.389399 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:19.389384    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8\": container with ID starting with 78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8 not found: ID does not exist" containerID="78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"
Oct 29 23:30:19.389446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.389406    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8"} err="failed to get container status \"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8\": rpc error: code = NotFound desc = could not find container \"78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8\": container with ID starting with 78ac6215f09acf3284cf6d84fc1bfa5faff604bf550206d6f8952e42ae8f94d8 not found: ID does not exist"
Oct 29 23:30:19.389446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.389422    5111 scope.go:122] "RemoveContainer" containerID="56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"
Oct 29 23:30:19.398588 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.398573    5111 scope.go:122] "RemoveContainer" containerID="56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"
Oct 29 23:30:19.398848 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:19.398827    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df\": container with ID starting with 56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df not found: ID does not exist" containerID="56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"
Oct 29 23:30:19.398925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.398852    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df"} err="failed to get container status \"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df\": rpc error: code = NotFound desc = could not find container \"56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df\": container with ID starting with 56afdd059d5d28e84a3bbf449679c1f5759b4b551616734e69555b2c92b955df not found: ID does not exist"
Oct 29 23:30:19.398925 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.398870    5111 scope.go:122] "RemoveContainer" containerID="c1f4bb300a65005927ed432ba30f5ebf352eb6e1ea27a0929ba5e014dac6c5ac"
Oct 29 23:30:19.412577 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.412559    5111 scope.go:122] "RemoveContainer" containerID="80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"
Oct 29 23:30:19.420673 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.420653    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:19.422100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.422080    5111 scope.go:122] "RemoveContainer" containerID="80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"
Oct 29 23:30:19.422374 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:19.422353    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e\": container with ID starting with 80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e not found: ID does not exist" containerID="80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"
Oct 29 23:30:19.422457 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.422382    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e"} err="failed to get container status \"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e\": rpc error: code = NotFound desc = could not find container \"80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e\": container with ID starting with 80b535c43a547cbf5c21420ef1b3b08998c033760a6b382f8b325b3d09ab305e not found: ID does not exist"
Oct 29 23:30:19.439868 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.439848    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:30:19.448443 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.448422    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-tl8pq"]
Oct 29 23:30:19.461965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.461944    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:19.487357 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.487324    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:30:19.496473 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.496448    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6"]
Oct 29 23:30:19.583392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.583354    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:30:19.589512 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.589486    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-clqc7"]
Oct 29 23:30:19.604458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.604120    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:19.707062 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.706950    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:19.720595 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.720572    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\") pod \"a553c687-39a1-4627-b013-bb58c0ce31da\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") "
Oct 29 23:30:19.720734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.720720    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\" (UniqueName: \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\") pod \"a553c687-39a1-4627-b013-bb58c0ce31da\" (UID: \"a553c687-39a1-4627-b013-bb58c0ce31da\") "
Oct 29 23:30:19.720960 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.720939    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca" pod "a553c687-39a1-4627-b013-bb58c0ce31da" (UID: "a553c687-39a1-4627-b013-bb58c0ce31da"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:19.721220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.721204    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a553c687-39a1-4627-b013-bb58c0ce31da-maas-api-service-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.722820 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.722797    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s" pod "a553c687-39a1-4627-b013-bb58c0ce31da" (UID: "a553c687-39a1-4627-b013-bb58c0ce31da"). InnerVolumeSpecName "kube-api-access-jh67s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:19.823000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.822945    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jh67s\" (UniqueName: \"kubernetes.io/projected/a553c687-39a1-4627-b013-bb58c0ce31da-kube-api-access-jh67s\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:19.850307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.850278    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:19.916618 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.916579    5111 patch_prober.go:29] interesting pod/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:30:19.916821 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:19.916638    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-313d9c-openshift-default-d98bdc757-4vfn6" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:30:20.397486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.397459    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"
Oct 29 23:30:20.397959 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.397495    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6" event={"ID":"a553c687-39a1-4627-b013-bb58c0ce31da","Type":"ContainerDied","Data":"835175c6a26ad9b32183f89a3b394e91737cf130e69ca8a8f952233dc3d70756"}
Oct 29 23:30:20.397959 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.397571    5111 scope.go:122] "RemoveContainer" containerID="88a514db5e8703f5b7f0606347c6310f1f56f8ca716205d19478ae7c862e138d"
Oct 29 23:30:20.513277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.513243    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:20.519684 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.519653    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-be8726-29888610-t28v6"]
Oct 29 23:30:20.581451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.581427    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:20.636626 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.636598    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\" (UniqueName: \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\") pod \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") "
Oct 29 23:30:20.636877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.636855    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\") pod \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\" (UID: \"034fe130-c336-433c-ab5b-4f40a0cd9fb5\") "
Oct 29 23:30:20.637290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.637260    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca" pod "034fe130-c336-433c-ab5b-4f40a0cd9fb5" (UID: "034fe130-c336-433c-ab5b-4f40a0cd9fb5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:20.637440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.637423    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/034fe130-c336-433c-ab5b-4f40a0cd9fb5-maas-api-service-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:20.639061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.639038    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf" pod "034fe130-c336-433c-ab5b-4f40a0cd9fb5" (UID: "034fe130-c336-433c-ab5b-4f40a0cd9fb5"). InnerVolumeSpecName "kube-api-access-gj8qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:20.738967 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.738933    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gj8qf\" (UniqueName: \"kubernetes.io/projected/034fe130-c336-433c-ab5b-4f40a0cd9fb5-kube-api-access-gj8qf\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:20.904797 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:20.904642    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:21.062155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.062125    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" path="/var/lib/kubelet/pods/0fe04ca7-5e9c-4738-9b70-59d000efe122/volumes"
Oct 29 23:30:21.062614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.062601    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" path="/var/lib/kubelet/pods/0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5/volumes"
Oct 29 23:30:21.062956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.062945    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" path="/var/lib/kubelet/pods/a553c687-39a1-4627-b013-bb58c0ce31da/volumes"
Oct 29 23:30:21.063671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.063657    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" path="/var/lib/kubelet/pods/b62997a5-7b30-4018-92d9-1d78d2430963/volumes"
Oct 29 23:30:21.408631 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.408609    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"
Oct 29 23:30:21.409089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.408649    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2" event={"ID":"034fe130-c336-433c-ab5b-4f40a0cd9fb5","Type":"ContainerDied","Data":"e5dc244293f037996cc8440e97040eff1c133fea817196a2528ceaf2dd995213"}
Oct 29 23:30:21.409089 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.408715    5111 scope.go:122] "RemoveContainer" containerID="228fe973ffe69250e0ed111eec9644543c4100ec1a1a2c8763160bf3a07d3c75"
Oct 29 23:30:21.463879 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.463855    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:21.476999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:21.476958    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-837245-29888610-xznv2"]
Oct 29 23:30:22.424982 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:22.424938    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:23.066649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:23.066617    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" path="/var/lib/kubelet/pods/034fe130-c336-433c-ab5b-4f40a0cd9fb5/volumes"
Oct 29 23:30:23.871080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:23.871049    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:25.324461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:25.324425    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:26.148107 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:26.148070    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:30:26.560703 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.560625    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"]
Oct 29 23:30:26.561199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561184    5111 cpu_manager.go:414] "containerMap: removing container" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy"
Oct 29 23:30:26.561243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561199    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy"
Oct 29 23:30:26.561243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561217    5111 cpu_manager.go:414] "containerMap: removing container" podUID="ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" containerName="revoke-keys"
Oct 29 23:30:26.561243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561222    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" containerName="revoke-keys"
Oct 29 23:30:26.561243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561232    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561238    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561249    5111 cpu_manager.go:414] "containerMap: removing container" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561254    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561263    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f87007b6-23f4-402a-9459-62dfb6254f7e" containerName="revoke-keys"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561268    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f87007b6-23f4-402a-9459-62dfb6254f7e" containerName="revoke-keys"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561275    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561281    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561287    5111 cpu_manager.go:414] "containerMap: removing container" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561292    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561305    5111 cpu_manager.go:414] "containerMap: removing container" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561309    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561325    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561330    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561339    5111 cpu_manager.go:414] "containerMap: removing container" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy"
Oct 29 23:30:26.561402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561343    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561418    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561425    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" containerName="revoke-keys"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561430    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561435    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="b62997a5-7b30-4018-92d9-1d78d2430963" containerName="istio-proxy"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561443    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="0fe04ca7-5e9c-4738-9b70-59d000efe122" containerName="istio-proxy"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561452    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561456    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561461    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ffa5ed3-45b7-4e7e-a847-1fb002bf47c5" containerName="istio-proxy"
Oct 29 23:30:26.561800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.561466    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f87007b6-23f4-402a-9459-62dfb6254f7e" containerName="revoke-keys"
Oct 29 23:30:26.565197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.565175    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.585307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.585280    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"]
Oct 29 23:30:26.597449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.593050    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:30:26.597449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.593185    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:30:26.597449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.595479    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:30:26.597449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.595592    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:30:26.693822 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.693790    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:26.707461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.707422    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmrr5\" (UniqueName: \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.707617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.707548    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.707685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.707624    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.813053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.809381    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.813053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.809524    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.813053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.810356    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.813053 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.810578    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmrr5\" (UniqueName: \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.827165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.817022    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.838707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.838618    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmrr5\" (UniqueName: \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\") pod \"maas-api-revoke-sub-e3049769-qn7j8\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:26.880794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:26.880399    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:27.007992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.007936    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"]
Oct 29 23:30:27.008849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.008826    5111 cpu_manager.go:414] "containerMap: removing container" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:27.008918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.008850    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:27.009035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.009011    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="034fe130-c336-433c-ab5b-4f40a0cd9fb5" containerName="cleanup"
Oct 29 23:30:27.009035 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.009032    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a553c687-39a1-4627-b013-bb58c0ce31da" containerName="cleanup"
Oct 29 23:30:27.015885 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.015110    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.039484 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.039452    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"]
Oct 29 23:30:27.117450 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.115738    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9fch\" (UniqueName: \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.117450 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.115856    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.117450 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.116053    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.212045 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:27.211931    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed8c2c5_f561_4d5a_9e19_d7204c2e855c.slice/crio-742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5 WatchSource:0}: Error finding container 742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5: Status 404 returned error can't find the container with id 742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5
Oct 29 23:30:27.212600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.212563    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"]
Oct 29 23:30:27.217709 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.217677    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.219627 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.217824    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9fch\" (UniqueName: \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.219627 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.217913    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.222165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.222084    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.222846 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.222825    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.243557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.243527    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9fch\" (UniqueName: \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\") pod \"maas-api-revoke-sub-dae7dd00-dtm2w\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.273651 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.273612    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:27.286377 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.286340    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main" containerID="cri-o://b3c42c4741c2ab652d112a120ec5c552c53c4b369496e1ca5d542c7b1efad943" gracePeriod=45
Oct 29 23:30:27.287105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.287071    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" containerID="cri-o://b02794c01ca2a807d9e10dfb8146795e641433c3818942c92ff3d22173feb482" gracePeriod=45
Oct 29 23:30:27.288597 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.288563    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main" containerID="cri-o://aeac2d49973b890d42522fa7abf275773e0e08a789e37e587c3dbd272350c739" gracePeriod=45
Oct 29 23:30:27.357516 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.357489    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:27.454887 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.454853    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:27.470897 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.470843    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8" event={"ID":"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c","Type":"ContainerStarted","Data":"742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5"}
Oct 29 23:30:27.473573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.473544    5111 generic.go:356] "Generic (PLEG): container finished" podID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerID="aeac2d49973b890d42522fa7abf275773e0e08a789e37e587c3dbd272350c739" exitCode=0
Oct 29 23:30:27.473836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.473815    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" event={"ID":"f44ed094-369c-43e7-b834-1b3f071c4a65","Type":"ContainerDied","Data":"aeac2d49973b890d42522fa7abf275773e0e08a789e37e587c3dbd272350c739"}
Oct 29 23:30:27.476110 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.476087    5111 generic.go:356] "Generic (PLEG): container finished" podID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerID="b3c42c4741c2ab652d112a120ec5c552c53c4b369496e1ca5d542c7b1efad943" exitCode=0
Oct 29 23:30:27.476371 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.476353    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" event={"ID":"cfad574b-51c6-4b86-8a78-cf3d99faa747","Type":"ContainerDied","Data":"b3c42c4741c2ab652d112a120ec5c552c53c4b369496e1ca5d542c7b1efad943"}
Oct 29 23:30:27.478332 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.478312    5111 generic.go:356] "Generic (PLEG): container finished" podID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerID="b02794c01ca2a807d9e10dfb8146795e641433c3818942c92ff3d22173feb482" exitCode=0
Oct 29 23:30:27.479448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.479425    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" event={"ID":"fcc2829d-357a-4385-9fcd-afdff00793eb","Type":"ContainerDied","Data":"b02794c01ca2a807d9e10dfb8146795e641433c3818942c92ff3d22173feb482"}
Oct 29 23:30:27.535169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.535126    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": dial tcp 10.132.0.68:8000: connect: connection refused"
Oct 29 23:30:27.597899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.597872    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:27.673036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.673008    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"]
Oct 29 23:30:27.674423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.674398    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:30:27.689351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.689324    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:30:27.692690 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.692666    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:30:27.831732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.831664    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\" (UniqueName: \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.831791    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.832008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.831856    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\" (UniqueName: \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.831895    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.831962    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\" (UniqueName: \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.832202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832050    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832113    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832179    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832269    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832390    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.832522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832446    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832495    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832523    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\" (UniqueName: \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832569    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.832628 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832612    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.832841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832645    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\") pod \"f44ed094-369c-43e7-b834-1b3f071c4a65\" (UID: \"f44ed094-369c-43e7-b834-1b3f071c4a65\") "
Oct 29 23:30:27.832841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832707    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\") pod \"cfad574b-51c6-4b86-8a78-cf3d99faa747\" (UID: \"cfad574b-51c6-4b86-8a78-cf3d99faa747\") "
Oct 29 23:30:27.832841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.832736    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\" (UniqueName: \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\") pod \"fcc2829d-357a-4385-9fcd-afdff00793eb\" (UID: \"fcc2829d-357a-4385-9fcd-afdff00793eb\") "
Oct 29 23:30:27.833217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.833159    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.833376 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.833347    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.833595 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.833559    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.833595 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.833566    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.833789 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.833766    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.835316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835284    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "kube-api-access-6k679". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:27.835533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835508    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.835533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835510    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.835658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835582    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.835658 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835601    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.835775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835691    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.835839 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.835816    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.836138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.836110    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.837448 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.837409    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "kube-api-access-dbfpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:27.837556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.837534    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.844015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.843578    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs" pod "fcc2829d-357a-4385-9fcd-afdff00793eb" (UID: "fcc2829d-357a-4385-9fcd-afdff00793eb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:27.844015 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.843623    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:27.844333 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.844292    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:27.844484 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.844465    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs" pod "f44ed094-369c-43e7-b834-1b3f071c4a65" (UID: "f44ed094-369c-43e7-b834-1b3f071c4a65"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:27.846226 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.846200    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh" pod "cfad574b-51c6-4b86-8a78-cf3d99faa747" (UID: "cfad574b-51c6-4b86-8a78-cf3d99faa747"). InnerVolumeSpecName "kube-api-access-kjfzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936704    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936733    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjfzh\" (UniqueName: \"kubernetes.io/projected/cfad574b-51c6-4b86-8a78-cf3d99faa747-kube-api-access-kjfzh\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936744    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936752    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936760    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936768    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcc2829d-357a-4385-9fcd-afdff00793eb-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.936775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936775    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k679\" (UniqueName: \"kubernetes.io/projected/f44ed094-369c-43e7-b834-1b3f071c4a65-kube-api-access-6k679\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936783    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936799    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cfad574b-51c6-4b86-8a78-cf3d99faa747-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936806    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936814    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbfpx\" (UniqueName: \"kubernetes.io/projected/fcc2829d-357a-4385-9fcd-afdff00793eb-kube-api-access-dbfpx\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936821    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f44ed094-369c-43e7-b834-1b3f071c4a65-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936829    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f44ed094-369c-43e7-b834-1b3f071c4a65-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936836    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936843    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cfad574b-51c6-4b86-8a78-cf3d99faa747-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:27.937167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:27.936850    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcc2829d-357a-4385-9fcd-afdff00793eb-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:28.004343 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.004305    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:28.488074 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.488032    5111 generic.go:356] "Generic (PLEG): container finished" podID="7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" containerID="8dc97134f99b019dc15057f25fed4075ff436889e35553dd46c5f96a605cd12e" exitCode=0
Oct 29 23:30:28.488232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.488104    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8" event={"ID":"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c","Type":"ContainerDied","Data":"8dc97134f99b019dc15057f25fed4075ff436889e35553dd46c5f96a605cd12e"}
Oct 29 23:30:28.489694 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.489675    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"
Oct 29 23:30:28.489824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.489720    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg" event={"ID":"f44ed094-369c-43e7-b834-1b3f071c4a65","Type":"ContainerDied","Data":"64198c47e44b6c3831ded95e41469654c77f9107300cc9556908f6b098fb4ba3"}
Oct 29 23:30:28.489824 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.489754    5111 scope.go:122] "RemoveContainer" containerID="aeac2d49973b890d42522fa7abf275773e0e08a789e37e587c3dbd272350c739"
Oct 29 23:30:28.492315 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.492292    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"
Oct 29 23:30:28.492451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.492330    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9" event={"ID":"cfad574b-51c6-4b86-8a78-cf3d99faa747","Type":"ContainerDied","Data":"a0e3d6b9089778e055d8e5a82ecfd7077df71d51e047097653da573e8ec6ba7c"}
Oct 29 23:30:28.494258 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.494216    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb" event={"ID":"fcc2829d-357a-4385-9fcd-afdff00793eb","Type":"ContainerDied","Data":"c6a6809ac5dbae75f6db67c63a09db1ff57674136d2183779a34f38b55ab60bf"}
Oct 29 23:30:28.494624 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.494214    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"
Oct 29 23:30:28.496965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.496284    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w" event={"ID":"1c2f2eed-a323-42ab-abd4-81d2e24483ed","Type":"ContainerStarted","Data":"a389b8f479c3effc6f8d4b4b11f8181f22d3c5f7072483210285eb73d4616bb6"}
Oct 29 23:30:28.496965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.496327    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w" event={"ID":"1c2f2eed-a323-42ab-abd4-81d2e24483ed","Type":"ContainerStarted","Data":"bff746ee7a594bdb39eee7d5ef99c13b2efe54f591052849d4e234285c5fd19f"}
Oct 29 23:30:28.506750 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.506623    5111 scope.go:122] "RemoveContainer" containerID="b3c42c4741c2ab652d112a120ec5c552c53c4b369496e1ca5d542c7b1efad943"
Oct 29 23:30:28.519655 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.519632    5111 scope.go:122] "RemoveContainer" containerID="b02794c01ca2a807d9e10dfb8146795e641433c3818942c92ff3d22173feb482"
Oct 29 23:30:28.555469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.555412    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main" containerID="cri-o://a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567" gracePeriod=45
Oct 29 23:30:28.582144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.582120    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:30:28.597157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.597133    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/simulator-w0-2670a5-kserve-6fc866d64-29tqg"]
Oct 29 23:30:28.636752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.636733    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:30:28.642758 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.642734    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/premium-w0-2670a5-kserve-97dbfd669-5gtk9"]
Oct 29 23:30:28.696731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.696680    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w" podStartSLOduration=2.696664339 podStartE2EDuration="2.696664339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:28.694050668 +0000 UTC m=+1039.780333801" watchObservedRunningTime="2026-10-29 23:30:28.696664339 +0000 UTC m=+1039.782947511"
Oct 29 23:30:28.739269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.739237    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:30:28.745197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.745172    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2670a5/distinct-2-w0-2670a5-kserve-d68f6f795-4mcrb"]
Oct 29 23:30:28.846256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.846232    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:30:28.949486 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949458    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\" (UniqueName: \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.949675 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949522    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\" (UniqueName: \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.949675 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949600    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.949675 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949638    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.949854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949687    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.949854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.949719    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\") pod \"75f092d8-f18c-4cca-8c73-48ece7d668f9\" (UID: \"75f092d8-f18c-4cca-8c73-48ece7d668f9\") "
Oct 29 23:30:28.950112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.950067    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:28.950221 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.950199    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:28.950458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.950423    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:28.950610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.950591    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:28.952118 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.952094    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:28.952402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.952379    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "kube-api-access-ltjz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:28.962764 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:28.962741    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm" pod "75f092d8-f18c-4cca-8c73-48ece7d668f9" (UID: "75f092d8-f18c-4cca-8c73-48ece7d668f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:29.051731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.051695    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.051731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.051727    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.051731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.051742    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75f092d8-f18c-4cca-8c73-48ece7d668f9-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.052056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.051752    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f092d8-f18c-4cca-8c73-48ece7d668f9-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.052056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.051761    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltjz4\" (UniqueName: \"kubernetes.io/projected/75f092d8-f18c-4cca-8c73-48ece7d668f9-kube-api-access-ltjz4\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.062614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.062578    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" path="/var/lib/kubelet/pods/cfad574b-51c6-4b86-8a78-cf3d99faa747/volumes"
Oct 29 23:30:29.063092 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.063071    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" path="/var/lib/kubelet/pods/f44ed094-369c-43e7-b834-1b3f071c4a65/volumes"
Oct 29 23:30:29.063491 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.063475    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" path="/var/lib/kubelet/pods/fcc2829d-357a-4385-9fcd-afdff00793eb/volumes"
Oct 29 23:30:29.301425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.301377    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:29.506947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.506864    5111 generic.go:356] "Generic (PLEG): container finished" podID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerID="a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567" exitCode=0
Oct 29 23:30:29.506947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.506908    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"
Oct 29 23:30:29.507191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.506951    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" event={"ID":"75f092d8-f18c-4cca-8c73-48ece7d668f9","Type":"ContainerDied","Data":"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"}
Oct 29 23:30:29.507191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.507005    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8" event={"ID":"75f092d8-f18c-4cca-8c73-48ece7d668f9","Type":"ContainerDied","Data":"d1ecda015ba51575fdbccd15ad51133fd1e2a5372b00a885e7d2e66d2327c11d"}
Oct 29 23:30:29.507191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.507026    5111 scope.go:122] "RemoveContainer" containerID="a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"
Oct 29 23:30:29.510025 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.509998    5111 generic.go:356] "Generic (PLEG): container finished" podID="1c2f2eed-a323-42ab-abd4-81d2e24483ed" containerID="a389b8f479c3effc6f8d4b4b11f8181f22d3c5f7072483210285eb73d4616bb6" exitCode=0
Oct 29 23:30:29.510122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.510075    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w" event={"ID":"1c2f2eed-a323-42ab-abd4-81d2e24483ed","Type":"ContainerDied","Data":"a389b8f479c3effc6f8d4b4b11f8181f22d3c5f7072483210285eb73d4616bb6"}
Oct 29 23:30:29.519058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.519035    5111 scope.go:122] "RemoveContainer" containerID="a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"
Oct 29 23:30:29.519338 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:29.519314    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567\": container with ID starting with a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567 not found: ID does not exist" containerID="a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"
Oct 29 23:30:29.519401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.519347    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567"} err="failed to get container status \"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567\": rpc error: code = NotFound desc = could not find container \"a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567\": container with ID starting with a4428e5b2c4bff3063d8a1214612eba33c37068090320a833cf60c87ff2c3567 not found: ID does not exist"
Oct 29 23:30:29.579592 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.579560    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:30:29.585765 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.584070    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-837245/simulator-w6-837245-kserve-6b7687ccbd-xx2s8"]
Oct 29 23:30:29.658172 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.658142    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:29.759731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.759662    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\" (UniqueName: \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\") pod \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") "
Oct 29 23:30:29.759731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.759711    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\") pod \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") "
Oct 29 23:30:29.759904 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.759761    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\") pod \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\" (UID: \"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c\") "
Oct 29 23:30:29.760173 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.760145    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca" pod "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" (UID: "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:29.760355 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.760336    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.761778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.761756    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca" pod "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" (UID: "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:29.761778 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.761768    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5" pod "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" (UID: "7ed8c2c5-f561-4d5a-9e19-d7204c2e855c"). InnerVolumeSpecName "kube-api-access-pmrr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:29.861834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.861804    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:29.861834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:29.861833    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmrr5\" (UniqueName: \"kubernetes.io/projected/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c-kube-api-access-pmrr5\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:30.518002 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.517951    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"
Oct 29 23:30:30.518197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.518000    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8" event={"ID":"7ed8c2c5-f561-4d5a-9e19-d7204c2e855c","Type":"ContainerDied","Data":"742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5"}
Oct 29 23:30:30.518197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.518065    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="742e47c33e47ce68ba61596ac813ec4f61b97f757f976e2b4d9bdc85631a1dd5"
Oct 29 23:30:30.670563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.670535    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:30.769372 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.769303    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\") pod \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") "
Oct 29 23:30:30.769372 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.769344    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\" (UniqueName: \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\") pod \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") "
Oct 29 23:30:30.769535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.769393    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\") pod \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\" (UID: \"1c2f2eed-a323-42ab-abd4-81d2e24483ed\") "
Oct 29 23:30:30.769790 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.769766    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca" pod "1c2f2eed-a323-42ab-abd4-81d2e24483ed" (UID: "1c2f2eed-a323-42ab-abd4-81d2e24483ed"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:30.771459 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.771432    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca" pod "1c2f2eed-a323-42ab-abd4-81d2e24483ed" (UID: "1c2f2eed-a323-42ab-abd4-81d2e24483ed"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:30:30.771553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.771487    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch" pod "1c2f2eed-a323-42ab-abd4-81d2e24483ed" (UID: "1c2f2eed-a323-42ab-abd4-81d2e24483ed"). InnerVolumeSpecName "kube-api-access-m9fch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:30.816409 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.816368    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:30.871220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.871156    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:30.871220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.871185    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9fch\" (UniqueName: \"kubernetes.io/projected/1c2f2eed-a323-42ab-abd4-81d2e24483ed-kube-api-access-m9fch\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:30.871220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:30.871201    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/1c2f2eed-a323-42ab-abd4-81d2e24483ed-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:31.063094 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:31.063023    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" path="/var/lib/kubelet/pods/75f092d8-f18c-4cca-8c73-48ece7d668f9/volumes"
Oct 29 23:30:31.527395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:31.527369    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"
Oct 29 23:30:31.527547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:31.527424    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w" event={"ID":"1c2f2eed-a323-42ab-abd4-81d2e24483ed","Type":"ContainerDied","Data":"bff746ee7a594bdb39eee7d5ef99c13b2efe54f591052849d4e234285c5fd19f"}
Oct 29 23:30:31.527547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:31.527462    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bff746ee7a594bdb39eee7d5ef99c13b2efe54f591052849d4e234285c5fd19f"
Oct 29 23:30:32.221680 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:32.221647    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:33.538853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:33.538821    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:33.825381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:33.825349    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:34.014553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:34.014521    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:34.300048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:34.299623    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:34.470114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:34.470075    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:35.213057 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.213023    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:35.451265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.451224    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:35.701856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.701817    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:35.855237 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.855204    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:30:35.855532 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.855502    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy" containerID="cri-o://a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b" gracePeriod=30
Oct 29 23:30:35.866345 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.866320    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:30:35.866576 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.866549    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy" containerID="cri-o://b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f" gracePeriod=30
Oct 29 23:30:35.875105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.875083    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:30:35.875287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.875265    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy" containerID="cri-o://389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306" gracePeriod=30
Oct 29 23:30:35.915293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:35.915248    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:36.990368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:36.990337    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:38.737127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:38.737085    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:40.368400 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:40.368356    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:41.148477 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.148294    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:30:41.186295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.186269    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:30:41.221358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221032    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.221358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221125    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.221358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221184    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.221358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221274    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.221358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221316    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.222061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221401    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.222061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221431    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.222061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221499    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.222061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.221550    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\") pod \"016689f1-e9d0-40dd-a847-7607f990d01a\" (UID: \"016689f1-e9d0-40dd-a847-7607f990d01a\") "
Oct 29 23:30:41.223080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.222405    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:41.223080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.222675    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.223080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.222853    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.223080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.223057    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.224404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.224296    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.226207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.226179    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "kube-api-access-qxslj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.226295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.226270    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.226425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.226406    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:41.226635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.226607    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token" pod "016689f1-e9d0-40dd-a847-7607f990d01a" (UID: "016689f1-e9d0-40dd-a847-7607f990d01a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.227170 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.227149    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:30:41.322773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322743    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.322940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322792    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.322940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322809    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.322940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322840    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.322940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322895    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.322940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322924    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.322949    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323058    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323104    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323121    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323125    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323111    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.323262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323152    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323327    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.323620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323462    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.323620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323548    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323575    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:41.323620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323595    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323642    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323648    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323682    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323734    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323776    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\") pod \"0f581c36-9808-47b8-9665-19b8d242ab62\" (UID: \"0f581c36-9808-47b8-9665-19b8d242ab62\") "
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323786    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.323876 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.323806    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\") pod \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\" (UID: \"0a4034b6-6cab-4dbd-85bd-3d4a53749064\") "
Oct 29 23:30:41.324411 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324386    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324417    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324431    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/016689f1-e9d0-40dd-a847-7607f990d01a-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324444    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324459    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxslj\" (UniqueName: \"kubernetes.io/projected/016689f1-e9d0-40dd-a847-7607f990d01a-kube-api-access-qxslj\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324473    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324487    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324532    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/016689f1-e9d0-40dd-a847-7607f990d01a-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324547    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324560    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324572    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324585    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324598    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324611    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f581c36-9808-47b8-9665-19b8d242ab62-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324624    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/016689f1-e9d0-40dd-a847-7607f990d01a-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.324811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324635    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.325282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.324873    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.325282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.325090    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.325282 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.325263    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.326095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.326059    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.326214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.326079    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "kube-api-access-97qkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.326285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.326259    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.326913 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.326875    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "kube-api-access-d2xbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:41.327582 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.327553    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:41.327683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.327649    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:41.327936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.327910    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy" pod "0a4034b6-6cab-4dbd-85bd-3d4a53749064" (UID: "0a4034b6-6cab-4dbd-85bd-3d4a53749064"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.328456 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.328435    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy" pod "0f581c36-9808-47b8-9665-19b8d242ab62" (UID: "0f581c36-9808-47b8-9665-19b8d242ab62"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:41.426342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426256    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.426342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426295    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.426342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426311    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2xbc\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-kube-api-access-d2xbc\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.426342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426326    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.426342 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426340    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a4034b6-6cab-4dbd-85bd-3d4a53749064-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426353    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97qkz\" (UniqueName: \"kubernetes.io/projected/0a4034b6-6cab-4dbd-85bd-3d4a53749064-kube-api-access-97qkz\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426366    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f581c36-9808-47b8-9665-19b8d242ab62-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426378    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f581c36-9808-47b8-9665-19b8d242ab62-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426390    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426402    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f581c36-9808-47b8-9665-19b8d242ab62-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.427019 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.426415    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a4034b6-6cab-4dbd-85bd-3d4a53749064-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:41.625826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.625792    5111 generic.go:356] "Generic (PLEG): container finished" podID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerID="b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f" exitCode=0
Oct 29 23:30:41.626046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.625839    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"
Oct 29 23:30:41.626046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.625873    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" event={"ID":"0a4034b6-6cab-4dbd-85bd-3d4a53749064","Type":"ContainerDied","Data":"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"}
Oct 29 23:30:41.626046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.625938    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46" event={"ID":"0a4034b6-6cab-4dbd-85bd-3d4a53749064","Type":"ContainerDied","Data":"8d56f95b191a30431f6a0f0b90a1fd3e19b1db2a39d01f372c0c87db7320eb9e"}
Oct 29 23:30:41.626046 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.625969    5111 scope.go:122] "RemoveContainer" containerID="b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"
Oct 29 23:30:41.627289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.627271    5111 generic.go:356] "Generic (PLEG): container finished" podID="0f581c36-9808-47b8-9665-19b8d242ab62" containerID="389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306" exitCode=0
Oct 29 23:30:41.627392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.627324    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"
Oct 29 23:30:41.627392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.627353    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" event={"ID":"0f581c36-9808-47b8-9665-19b8d242ab62","Type":"ContainerDied","Data":"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"}
Oct 29 23:30:41.627493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.627409    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc" event={"ID":"0f581c36-9808-47b8-9665-19b8d242ab62","Type":"ContainerDied","Data":"1aa18bcd01504f2c21911b41a8aee93a576c076e169e516192f922cbb4734c72"}
Oct 29 23:30:41.628889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.628865    5111 generic.go:356] "Generic (PLEG): container finished" podID="016689f1-e9d0-40dd-a847-7607f990d01a" containerID="a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b" exitCode=0
Oct 29 23:30:41.629036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.628916    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" event={"ID":"016689f1-e9d0-40dd-a847-7607f990d01a","Type":"ContainerDied","Data":"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"}
Oct 29 23:30:41.629036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.628927    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"
Oct 29 23:30:41.629036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.628953    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv" event={"ID":"016689f1-e9d0-40dd-a847-7607f990d01a","Type":"ContainerDied","Data":"7198024a96ee62f48f3017ac1799f3beefe19be3b8189d72fe5d4d1c69ca346a"}
Oct 29 23:30:41.637688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.637670    5111 scope.go:122] "RemoveContainer" containerID="b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"
Oct 29 23:30:41.637961 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:41.637936    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f\": container with ID starting with b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f not found: ID does not exist" containerID="b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"
Oct 29 23:30:41.638092 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.638008    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f"} err="failed to get container status \"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f\": rpc error: code = NotFound desc = could not find container \"b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f\": container with ID starting with b401ce9e6c0f538f073a1ad8cce2b77118ca0b5b1ae02b478eba869676aaa95f not found: ID does not exist"
Oct 29 23:30:41.638092 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.638031    5111 scope.go:122] "RemoveContainer" containerID="389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"
Oct 29 23:30:41.648062 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.648046    5111 scope.go:122] "RemoveContainer" containerID="389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"
Oct 29 23:30:41.648311 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:41.648293    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306\": container with ID starting with 389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306 not found: ID does not exist" containerID="389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"
Oct 29 23:30:41.648364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.648317    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306"} err="failed to get container status \"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306\": rpc error: code = NotFound desc = could not find container \"389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306\": container with ID starting with 389af6c4be6600c51f2e30d116d30461c930dfd2858cd6864d92c6ad0c5ff306 not found: ID does not exist"
Oct 29 23:30:41.648364 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.648334    5111 scope.go:122] "RemoveContainer" containerID="a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"
Oct 29 23:30:41.656741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.656722    5111 scope.go:122] "RemoveContainer" containerID="a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"
Oct 29 23:30:41.657013 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:41.656993    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b\": container with ID starting with a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b not found: ID does not exist" containerID="a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"
Oct 29 23:30:41.657073 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.657018    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b"} err="failed to get container status \"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b\": rpc error: code = NotFound desc = could not find container \"a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b\": container with ID starting with a49f30c55a65081a5a4fa512b2d70f374b222c4d534d2ce14f61d8a0d6be2d4b not found: ID does not exist"
Oct 29 23:30:41.662596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.662577    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:41.685381 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.685331    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:30:41.698949 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.698929    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-b4mlc"]
Oct 29 23:30:41.727596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.727573    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:30:41.731138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.731118    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-ckb46"]
Oct 29 23:30:41.766881 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.766857    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:30:41.772175 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.772154    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-032fa8-openshift-default-7b6d48f8fc-bqjzv"]
Oct 29 23:30:41.887743 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:41.887710    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:43.062293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:43.062257    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" path="/var/lib/kubelet/pods/016689f1-e9d0-40dd-a847-7607f990d01a/volumes"
Oct 29 23:30:43.062717 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:43.062703    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" path="/var/lib/kubelet/pods/0a4034b6-6cab-4dbd-85bd-3d4a53749064/volumes"
Oct 29 23:30:43.063184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:43.063169    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" path="/var/lib/kubelet/pods/0f581c36-9808-47b8-9665-19b8d242ab62/volumes"
Oct 29 23:30:43.478254 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:43.478222    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:44.648302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:44.648263    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:44.990043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:44.989462    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:45.150386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.150345    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:45.302106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.302016    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:45.314107 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.314079    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"]
Oct 29 23:30:45.329552 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.329526    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-837245-dfvl2"]
Oct 29 23:30:45.731728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.731694    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:45.791328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.791299    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"]
Oct 29 23:30:45.804702 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.804677    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-2670a5-9vg9s"]
Oct 29 23:30:45.877325 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:45.877271    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:46.336473 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:46.336437    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:47.061465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:47.061431    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60a5bb98-9467-40b4-91ef-7adc9e570647" path="/var/lib/kubelet/pods/60a5bb98-9467-40b4-91ef-7adc9e570647/volumes"
Oct 29 23:30:47.061883 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:47.061868    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="65361eaa-48a4-442a-92d1-a090fefbb3f5" path="/var/lib/kubelet/pods/65361eaa-48a4-442a-92d1-a090fefbb3f5/volumes"
Oct 29 23:30:47.434346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:47.434316    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:48.943202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:48.943156    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:49.094430 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:49.094386    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:50.508164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:50.508129    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:51.225752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:51.225716    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:52.216575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:52.216540    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:52.385292 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:52.385254    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:52.530727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:52.530590    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:53.369017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.368964    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:53.549669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.549625    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:53.786727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.786638    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:30:53.787194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.787159    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy" containerID="cri-o://a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f" gracePeriod=30
Oct 29 23:30:53.840792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.840750    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:30:53.878999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.874789    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:30:53.878999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.875040    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy" containerID="cri-o://b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba" gracePeriod=30
Oct 29 23:30:53.878999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.869655    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:53.878999 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:53.875543    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy" containerID="cri-o://53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c" gracePeriod=30
Oct 29 23:30:54.316086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.316050    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:54.457743 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.457712    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:54.597139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.597104    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:30:54.597384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.597353    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy" containerID="cri-o://5dfa1f60f9f0db1095bab37b70b30cdf3e7a09ed867ed4871170346e1b737b77" gracePeriod=30
Oct 29 23:30:54.613359 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.613300    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:54.619351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.619314    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:30:54.619584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.619557    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy" containerID="cri-o://3790c0facfb272ead08611739289c5ed1ad0d08faf4ebcd99963f0fb13c060e5" gracePeriod=30
Oct 29 23:30:54.630838 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.630807    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:30:54.631059 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.631036    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy" containerID="cri-o://636b53ea06ae1fcefc90a2c2dff7bf2ad940d4438c2873efc321c66c299b18ae" gracePeriod=30
Oct 29 23:30:54.817591 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:54.817558    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:55.216370 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216337    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:30:55.216830 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216816    5111 cpu_manager.go:414] "containerMap: removing container" podUID="7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" containerName="revoke-keys"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216830    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" containerName="revoke-keys"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216842    5111 cpu_manager.go:414] "containerMap: removing container" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216847    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216859    5111 cpu_manager.go:414] "containerMap: removing container" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216864    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216869    5111 cpu_manager.go:414] "containerMap: removing container" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy"
Oct 29 23:30:55.216880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216875    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216888    5111 cpu_manager.go:414] "containerMap: removing container" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216892    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216907    5111 cpu_manager.go:414] "containerMap: removing container" podUID="1c2f2eed-a323-42ab-abd4-81d2e24483ed" containerName="revoke-keys"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216911    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c2f2eed-a323-42ab-abd4-81d2e24483ed" containerName="revoke-keys"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216928    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216932    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216944    5111 cpu_manager.go:414] "containerMap: removing container" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216948    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216954    5111 cpu_manager.go:414] "containerMap: removing container" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.216959    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217053    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="016689f1-e9d0-40dd-a847-7607f990d01a" containerName="istio-proxy"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217066    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f581c36-9808-47b8-9665-19b8d242ab62" containerName="istio-proxy"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217077    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="75f092d8-f18c-4cca-8c73-48ece7d668f9" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217083    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f44ed094-369c-43e7-b834-1b3f071c4a65" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217091    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c2f2eed-a323-42ab-abd4-81d2e24483ed" containerName="revoke-keys"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217097    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="cfad574b-51c6-4b86-8a78-cf3d99faa747" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217103    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="fcc2829d-357a-4385-9fcd-afdff00793eb" containerName="main"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217110    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" containerName="revoke-keys"
Oct 29 23:30:55.217123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.217116    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a4034b6-6cab-4dbd-85bd-3d4a53749064" containerName="istio-proxy"
Oct 29 23:30:55.222637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.222617    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.246458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.246421    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:30:55.290387 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290360    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290544 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290423    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lxgp\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290544 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290530    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290627 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290587    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290673    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290719    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290735    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290810    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.290867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.290855    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.391615 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391581    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.391793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391633    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.391793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391689    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.391793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391726    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.391793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391769    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392052 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391819    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392052 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391889    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392052 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.391944    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lxgp\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392052 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392037    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392218    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392352    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392447 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392424    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392635    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.392657 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.392650    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.394554 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.394524    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.394652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.394641    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.413243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.413218    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.413340 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.413297    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lxgp\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\") pod \"e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") " pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.538652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.538579    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:55.867574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.867544    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:30:55.867719 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:30:55.867698    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d1780e_f31a_4f2f_a0f1_ccd0d168f50c.slice/crio-8cc787db11faecde14ac52fd34f27b198847044c16bb13e0aea7d29079094c38 WatchSource:0}: Error finding container 8cc787db11faecde14ac52fd34f27b198847044c16bb13e0aea7d29079094c38: Status 404 returned error can't find the container with id 8cc787db11faecde14ac52fd34f27b198847044c16bb13e0aea7d29079094c38
Oct 29 23:30:55.870214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.870183    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:30:55.870287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.870245    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:30:55.870287 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:55.870275    5111 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Oct 29 23:30:56.163271 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:56.163186    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:30:56.197005 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:56.196961    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:56.818728 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:56.818685    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" event={"ID":"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c","Type":"ContainerStarted","Data":"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"}
Oct 29 23:30:56.819170 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:56.818782    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" event={"ID":"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c","Type":"ContainerStarted","Data":"8cc787db11faecde14ac52fd34f27b198847044c16bb13e0aea7d29079094c38"}
Oct 29 23:30:56.898404 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:56.898320    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podStartSLOduration=1.8982995649999999 podStartE2EDuration="1.898299565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:56.890514154 +0000 UTC m=+1067.976797369" watchObservedRunningTime="2026-10-29 23:30:56.898299565 +0000 UTC m=+1067.984582727"
Oct 29 23:30:57.344176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:57.344137    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:57.539124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:57.539093    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:30:57.540480 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:57.540453    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 29 23:30:57.540605 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:57.540513    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 29 23:30:57.543522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:57.543482    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:58.539289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:58.539251    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 29 23:30:58.539775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:58.539312    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 29 23:30:58.836048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:58.836014    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:30:59.141827 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.141797    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:30:59.220634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.220606    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:30:59.246352 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246318    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246372    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246419    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246442    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246511 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246475    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.246731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246514    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246567    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246615    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.246731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246658    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.246731 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246702    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246741    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246810    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246859    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246877    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246886    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.247014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.246966    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247062    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247105    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247131    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247217    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\") pod \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\" (UID: \"4dad75f8-a173-45a3-a7c9-cb49b639a64c\") "
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247253    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\") pod \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\" (UID: \"ed2716a9-be29-4ec9-8ab3-b0b4ffead918\") "
Oct 29 23:30:59.247331 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247307    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.247779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247755    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.247863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247786    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.247863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247800    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.247863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247812    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.247863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247760    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.247863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.247841    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:59.248142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.248107    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.248142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.248118    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.248492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.248294    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.248492 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.248360    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.250536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.250504    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.251090 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.251032    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "kube-api-access-zv7b2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.251939 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.251913    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:59.252300 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.252271    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "kube-api-access-8wkhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.252384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.252278    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:59.252384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.252285    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token" pod "ed2716a9-be29-4ec9-8ab3-b0b4ffead918" (UID: "ed2716a9-be29-4ec9-8ab3-b0b4ffead918"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.253212 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.253191    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.254398 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.254374    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token" pod "4dad75f8-a173-45a3-a7c9-cb49b639a64c" (UID: "4dad75f8-a173-45a3-a7c9-cb49b639a64c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.269334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.269315    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:30:59.348371 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348345    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348558 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348385    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348558 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348403    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348558 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348429    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348558 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348545    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348580    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348606    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348644    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348715    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\") pod \"45837438-49af-4dea-b0c6-0d0a64649b41\" (UID: \"45837438-49af-4dea-b0c6-0d0a64649b41\") "
Oct 29 23:30:59.348795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.348742    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.349070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349037    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349049    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349064    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349083    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349100    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349113    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349126    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv7b2\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-kube-api-access-zv7b2\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349139    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349152    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349166    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349178    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349190    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349201    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349213    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wkhq\" (UniqueName: \"kubernetes.io/projected/ed2716a9-be29-4ec9-8ab3-b0b4ffead918-kube-api-access-8wkhq\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.349230 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349226    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4dad75f8-a173-45a3-a7c9-cb49b639a64c-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.350027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349322    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:30:59.350027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349575    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.350027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349754    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.350027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.349938    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.351237 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.351206    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.351826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.351797    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:30:59.351916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.351830    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:30:59.352187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.352165    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs" pod "45837438-49af-4dea-b0c6-0d0a64649b41" (UID: "45837438-49af-4dea-b0c6-0d0a64649b41"). InnerVolumeSpecName "kube-api-access-8hjfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:30:59.450823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450790    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.450823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450818    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.450823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450831    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.451045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450839    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.451045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450848    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hjfs\" (UniqueName: \"kubernetes.io/projected/45837438-49af-4dea-b0c6-0d0a64649b41-kube-api-access-8hjfs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.451045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450858    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45837438-49af-4dea-b0c6-0d0a64649b41-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.451045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450866    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45837438-49af-4dea-b0c6-0d0a64649b41-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.451045 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.450874    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45837438-49af-4dea-b0c6-0d0a64649b41-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:30:59.539903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.539871    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 29 23:30:59.540311 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.539920    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 29 23:30:59.848034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.847033    5111 generic.go:356] "Generic (PLEG): container finished" podID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerID="636b53ea06ae1fcefc90a2c2dff7bf2ad940d4438c2873efc321c66c299b18ae" exitCode=0
Oct 29 23:30:59.848034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.847176    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" event={"ID":"1580d4e7-b803-4909-bf1c-317d4bc6224f","Type":"ContainerDied","Data":"636b53ea06ae1fcefc90a2c2dff7bf2ad940d4438c2873efc321c66c299b18ae"}
Oct 29 23:30:59.849754 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.849731    5111 generic.go:356] "Generic (PLEG): container finished" podID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerID="3790c0facfb272ead08611739289c5ed1ad0d08faf4ebcd99963f0fb13c060e5" exitCode=0
Oct 29 23:30:59.849885 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.849865    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" event={"ID":"e1c56655-f6ef-4073-96ee-7836832a74ee","Type":"ContainerDied","Data":"3790c0facfb272ead08611739289c5ed1ad0d08faf4ebcd99963f0fb13c060e5"}
Oct 29 23:30:59.852621 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.852598    5111 generic.go:356] "Generic (PLEG): container finished" podID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerID="53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c" exitCode=0
Oct 29 23:30:59.852701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.852667    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" event={"ID":"4dad75f8-a173-45a3-a7c9-cb49b639a64c","Type":"ContainerDied","Data":"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"}
Oct 29 23:30:59.852701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.852699    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5" event={"ID":"4dad75f8-a173-45a3-a7c9-cb49b639a64c","Type":"ContainerDied","Data":"7a05d6b3a34ac2558b9e136c1f2fe36d1509707915c8017a6eebe9a6f603526f"}
Oct 29 23:30:59.852779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.852719    5111 scope.go:122] "RemoveContainer" containerID="53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"
Oct 29 23:30:59.852854 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.852842    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"
Oct 29 23:30:59.864000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.860332    5111 generic.go:356] "Generic (PLEG): container finished" podID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerID="a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f" exitCode=0
Oct 29 23:30:59.864000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.860401    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" event={"ID":"ed2716a9-be29-4ec9-8ab3-b0b4ffead918","Type":"ContainerDied","Data":"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"}
Oct 29 23:30:59.864000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.860439    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz" event={"ID":"ed2716a9-be29-4ec9-8ab3-b0b4ffead918","Type":"ContainerDied","Data":"4aebcfa37f26ab7319b438a8b695b4826258e4177f4019977556844b92bfc08c"}
Oct 29 23:30:59.864000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.860512    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"
Oct 29 23:30:59.870224 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.870175    5111 generic.go:356] "Generic (PLEG): container finished" podID="309fd91b-71ee-4957-808b-5143e283a5d8" containerID="5dfa1f60f9f0db1095bab37b70b30cdf3e7a09ed867ed4871170346e1b737b77" exitCode=0
Oct 29 23:30:59.870439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.870305    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" event={"ID":"309fd91b-71ee-4957-808b-5143e283a5d8","Type":"ContainerDied","Data":"5dfa1f60f9f0db1095bab37b70b30cdf3e7a09ed867ed4871170346e1b737b77"}
Oct 29 23:30:59.903579 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.903554    5111 generic.go:356] "Generic (PLEG): container finished" podID="45837438-49af-4dea-b0c6-0d0a64649b41" containerID="b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba" exitCode=0
Oct 29 23:30:59.903811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.903611    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"
Oct 29 23:30:59.903907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.903627    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" event={"ID":"45837438-49af-4dea-b0c6-0d0a64649b41","Type":"ContainerDied","Data":"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"}
Oct 29 23:30:59.904031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.903918    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct" event={"ID":"45837438-49af-4dea-b0c6-0d0a64649b41","Type":"ContainerDied","Data":"57f1b6734388972103ab3a4bcb527a0d44cea07699963f39f6ab9bd33ad63a9f"}
Oct 29 23:30:59.906476 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.906453    5111 scope.go:122] "RemoveContainer" containerID="53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"
Oct 29 23:30:59.906867 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:59.906848    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c\": container with ID starting with 53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c not found: ID does not exist" containerID="53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"
Oct 29 23:30:59.906950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.906878    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c"} err="failed to get container status \"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c\": rpc error: code = NotFound desc = could not find container \"53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c\": container with ID starting with 53952d81446bb1760751dca815acce3079af223d7bd22bc1302e9547087a701c not found: ID does not exist"
Oct 29 23:30:59.906950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.906896    5111 scope.go:122] "RemoveContainer" containerID="a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"
Oct 29 23:30:59.918111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.918093    5111 scope.go:122] "RemoveContainer" containerID="a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"
Oct 29 23:30:59.918425 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:59.918395    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f\": container with ID starting with a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f not found: ID does not exist" containerID="a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"
Oct 29 23:30:59.918502 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.918436    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f"} err="failed to get container status \"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f\": rpc error: code = NotFound desc = could not find container \"a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f\": container with ID starting with a02f6dadad2de9e09640162ae5abc2527f43d873b4aef37e22768d69ec1f775f not found: ID does not exist"
Oct 29 23:30:59.918502 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.918458    5111 scope.go:122] "RemoveContainer" containerID="b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"
Oct 29 23:30:59.955841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.955818    5111 scope.go:122] "RemoveContainer" containerID="b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"
Oct 29 23:30:59.956495 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:30:59.956473    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba\": container with ID starting with b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba not found: ID does not exist" containerID="b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"
Oct 29 23:30:59.956604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.956502    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba"} err="failed to get container status \"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba\": rpc error: code = NotFound desc = could not find container \"b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba\": container with ID starting with b12a3c095f301140d4c9d6f238ad42205703e1543b0d8b7820651f536be448ba not found: ID does not exist"
Oct 29 23:30:59.967620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.967513    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:30:59.971773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:30:59.971570    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-nn7ct"]
Oct 29 23:31:00.044452 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.044417    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:31:00.045528 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.045492    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:31:00.048691 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.048640    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-flhvz"]
Oct 29 23:31:00.114513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.114489    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:31:00.117936 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.117918    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2670a5-openshift-default-9f9b57bbc-p82t5"]
Oct 29 23:31:00.122574 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.122523    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:31:00.125312 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.125294    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:31:00.157738 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157717    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.157853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157778    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.157853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157808    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.157853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157836    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157869    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157914    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.157952    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158049 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158015    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158046    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158079    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158108    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158144    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158174    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158202    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158283 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158232    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158299    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158338    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158382    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158403    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158410    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158420    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158494    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158527    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158583    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158606    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.158661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158616    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158682    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158741    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\") pod \"1580d4e7-b803-4909-bf1c-317d4bc6224f\" (UID: \"1580d4e7-b803-4909-bf1c-317d4bc6224f\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158781    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\") pod \"e1c56655-f6ef-4073-96ee-7836832a74ee\" (UID: \"e1c56655-f6ef-4073-96ee-7836832a74ee\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158847    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.158914    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\") pod \"309fd91b-71ee-4957-808b-5143e283a5d8\" (UID: \"309fd91b-71ee-4957-808b-5143e283a5d8\") "
Oct 29 23:31:00.159178 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.159172    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.159564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.159542    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.159696 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.159683    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.159823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.159812    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.159930 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.159918    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.160105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.160077    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.160276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.160125    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:00.160394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.160369    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.162519    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:00.163078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.162724    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.162837    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163088    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163109    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163301 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163190    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "kube-api-access-lgs6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.163463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163298    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163307    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163463 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.160096    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c56655-f6ef-4073-96ee-7836832a74ee-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.163616 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163483    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163616 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163568    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.163856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163828    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.163933 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163867    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert" pod "1580d4e7-b803-4909-bf1c-317d4bc6224f" (UID: "1580d4e7-b803-4909-bf1c-317d4bc6224f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:00.164021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.163960    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.164346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.164322    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "kube-api-access-pq6kk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.164942 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.164917    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:00.166231 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.166204    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.166680 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.166658    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:00.167329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.167301    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo" pod "e1c56655-f6ef-4073-96ee-7836832a74ee" (UID: "e1c56655-f6ef-4073-96ee-7836832a74ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:00.168016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.167995    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk" pod "309fd91b-71ee-4957-808b-5143e283a5d8" (UID: "309fd91b-71ee-4957-808b-5143e283a5d8"). InnerVolumeSpecName "kube-api-access-5wbrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:00.199098 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.198874    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:00.264646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264614    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264646 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264649    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgs6z\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-kube-api-access-lgs6z\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264666    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264679    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264693    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/309fd91b-71ee-4957-808b-5143e283a5d8-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264708    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264721    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264733    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264745    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264757    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1580d4e7-b803-4909-bf1c-317d4bc6224f-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264778    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/309fd91b-71ee-4957-808b-5143e283a5d8-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264791    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wbrk\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-kube-api-access-5wbrk\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264804    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq6kk\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-kube-api-access-pq6kk\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.264815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264818    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264831    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264843    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/309fd91b-71ee-4957-808b-5143e283a5d8-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264855    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264868    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264879    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264891    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/309fd91b-71ee-4957-808b-5143e283a5d8-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264904    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1580d4e7-b803-4909-bf1c-317d4bc6224f-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.265189 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.264917    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c56655-f6ef-4073-96ee-7836832a74ee-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:00.342033 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.341999    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:00.499054 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.496448    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:00.539228 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.539196    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 29 23:31:00.539413 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.539244    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 29 23:31:00.794556 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.794454    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:00.913748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.913719    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"
Oct 29 23:31:00.913748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.913735    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k" event={"ID":"309fd91b-71ee-4957-808b-5143e283a5d8","Type":"ContainerDied","Data":"ee6dbdf985d32d545121f29039b8dd1be62f097842bc298d4144115edc71cda1"}
Oct 29 23:31:00.914006 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.913796    5111 scope.go:122] "RemoveContainer" containerID="5dfa1f60f9f0db1095bab37b70b30cdf3e7a09ed867ed4871170346e1b737b77"
Oct 29 23:31:00.916545 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.916526    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"
Oct 29 23:31:00.916691 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.916569    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns" event={"ID":"1580d4e7-b803-4909-bf1c-317d4bc6224f","Type":"ContainerDied","Data":"011c40d9219a7da3cd9d2bbcc7966664b327c37fa35f41be2e2f9b6813c0848a"}
Oct 29 23:31:00.918407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.918305    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj" event={"ID":"e1c56655-f6ef-4073-96ee-7836832a74ee","Type":"ContainerDied","Data":"95beae5e5a2d12694be1ae0cbd5c4270be065b8a5de2457919c972fd03bffd5e"}
Oct 29 23:31:00.918407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.918370    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"
Oct 29 23:31:00.926470 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.926299    5111 scope.go:122] "RemoveContainer" containerID="636b53ea06ae1fcefc90a2c2dff7bf2ad940d4438c2873efc321c66c299b18ae"
Oct 29 23:31:00.937955 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:00.937931    5111 scope.go:122] "RemoveContainer" containerID="3790c0facfb272ead08611739289c5ed1ad0d08faf4ebcd99963f0fb13c060e5"
Oct 29 23:31:01.002052 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.002024    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:31:01.014510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.014479    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-dkvhj"]
Oct 29 23:31:01.038879 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.038854    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:31:01.042903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.042878    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-chtns"]
Oct 29 23:31:01.060879 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.060827    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" path="/var/lib/kubelet/pods/1580d4e7-b803-4909-bf1c-317d4bc6224f/volumes"
Oct 29 23:31:01.061339 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.061324    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" path="/var/lib/kubelet/pods/45837438-49af-4dea-b0c6-0d0a64649b41/volumes"
Oct 29 23:31:01.061748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.061729    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" path="/var/lib/kubelet/pods/4dad75f8-a173-45a3-a7c9-cb49b639a64c/volumes"
Oct 29 23:31:01.062749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.062725    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" path="/var/lib/kubelet/pods/e1c56655-f6ef-4073-96ee-7836832a74ee/volumes"
Oct 29 23:31:01.063242 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.063223    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" path="/var/lib/kubelet/pods/ed2716a9-be29-4ec9-8ab3-b0b4ffead918/volumes"
Oct 29 23:31:01.099243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.099212    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:31:01.103114 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.103091    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-837245-openshift-default-67857c7775-hcq4k"]
Oct 29 23:31:01.539888 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.539852    5111 patch_prober.go:29] interesting pod/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Oct 29 23:31:01.540087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:01.539921    5111 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Oct 29 23:31:02.089105 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:02.089071    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:02.633668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:02.633634    5111 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:31:02.634529 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:02.634497    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:31:02.635906 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:02.635883    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:31:03.061956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:03.061884    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" path="/var/lib/kubelet/pods/309fd91b-71ee-4957-808b-5143e283a5d8/volumes"
Oct 29 23:31:03.742317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:03.742283    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:05.170472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:05.170426    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:05.721643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:05.721611    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:31:05.735453 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:05.735420    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:31:05.852859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:05.852827    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:06.521167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:06.521138    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:07.827326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:07.827295    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:07.962583 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:07.962547    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:09.496579 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:09.496535    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:10.981634 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:10.981550    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:11.680257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:11.680220    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:11.824299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:11.824268    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:12.501202 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:12.501167    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:13.694953 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.694921    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"]
Oct 29 23:31:13.695402 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695387    5111 cpu_manager.go:414] "containerMap: removing container" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy"
Oct 29 23:31:13.695455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695404    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy"
Oct 29 23:31:13.695455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695424    5111 cpu_manager.go:414] "containerMap: removing container" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy"
Oct 29 23:31:13.695455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695431    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy"
Oct 29 23:31:13.695455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695444    5111 cpu_manager.go:414] "containerMap: removing container" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy"
Oct 29 23:31:13.695455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695451    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695463    5111 cpu_manager.go:414] "containerMap: removing container" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695468    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695476    5111 cpu_manager.go:414] "containerMap: removing container" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695480    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695495    5111 cpu_manager.go:414] "containerMap: removing container" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695503    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695587    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="45837438-49af-4dea-b0c6-0d0a64649b41" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695603    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dad75f8-a173-45a3-a7c9-cb49b639a64c" containerName="istio-proxy"
Oct 29 23:31:13.695608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695610    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="309fd91b-71ee-4957-808b-5143e283a5d8" containerName="istio-proxy"
Oct 29 23:31:13.695940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695618    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed2716a9-be29-4ec9-8ab3-b0b4ffead918" containerName="istio-proxy"
Oct 29 23:31:13.695940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695627    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="1580d4e7-b803-4909-bf1c-317d4bc6224f" containerName="istio-proxy"
Oct 29 23:31:13.695940 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.695634    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c56655-f6ef-4073-96ee-7836832a74ee" containerName="istio-proxy"
Oct 29 23:31:13.702547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.702522    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.719391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.719365    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"]
Oct 29 23:31:13.729643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.729617    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:31:13.729781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.729636    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:31:13.729781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.729674    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:31:13.732432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.732414    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:31:13.827638 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.827613    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.827747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.827657    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d55x\" (UniqueName: \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.827803 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.827783    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.928913 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.928883    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.929061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.928940    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.929061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.929000    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d55x\" (UniqueName: \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.929579 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.929554    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.931704 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.931685    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:13.949469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:13.949421    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d55x\" (UniqueName: \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\") pod \"maas-api-revoke-sub-26c2e2a2-9vw8z\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:14.017737 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:14.017710    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:14.051221 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:14.051183    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:14.333629 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:14.333602    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"]
Oct 29 23:31:14.334061 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:31:14.334042    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42551b05_0c68_471b_883d_c7df49c9f2cd.slice/crio-abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9 WatchSource:0}: Error finding container abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9: Status 404 returned error can't find the container with id abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9
Oct 29 23:31:15.023847 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:15.023818    5111 generic.go:356] "Generic (PLEG): container finished" podID="42551b05-0c68-471b-883d-c7df49c9f2cd" containerID="53ec5ff4231ac9a9f18c3e46918e945f0bb1e5edfaeb9c88af039b7f4884777c" exitCode=0
Oct 29 23:31:15.024269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:15.023943    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z" event={"ID":"42551b05-0c68-471b-883d-c7df49c9f2cd","Type":"ContainerDied","Data":"53ec5ff4231ac9a9f18c3e46918e945f0bb1e5edfaeb9c88af039b7f4884777c"}
Oct 29 23:31:15.024269 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:15.024017    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z" event={"ID":"42551b05-0c68-471b-883d-c7df49c9f2cd","Type":"ContainerStarted","Data":"abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9"}
Oct 29 23:31:15.561183 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:15.561137    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:16.162095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.162066    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:16.252777 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.252749    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\" (UniqueName: \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\") pod \"42551b05-0c68-471b-883d-c7df49c9f2cd\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") "
Oct 29 23:31:16.252903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.252786    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\") pod \"42551b05-0c68-471b-883d-c7df49c9f2cd\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") "
Oct 29 23:31:16.252903 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.252834    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\") pod \"42551b05-0c68-471b-883d-c7df49c9f2cd\" (UID: \"42551b05-0c68-471b-883d-c7df49c9f2cd\") "
Oct 29 23:31:16.253211 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.253186    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca" pod "42551b05-0c68-471b-883d-c7df49c9f2cd" (UID: "42551b05-0c68-471b-883d-c7df49c9f2cd"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:16.253299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.253287    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:16.254795 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.254771    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca" pod "42551b05-0c68-471b-883d-c7df49c9f2cd" (UID: "42551b05-0c68-471b-883d-c7df49c9f2cd"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:16.254880 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.254835    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x" pod "42551b05-0c68-471b-883d-c7df49c9f2cd" (UID: "42551b05-0c68-471b-883d-c7df49c9f2cd"). InnerVolumeSpecName "kube-api-access-6d55x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:16.354472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.354451    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d55x\" (UniqueName: \"kubernetes.io/projected/42551b05-0c68-471b-883d-c7df49c9f2cd-kube-api-access-6d55x\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:16.354472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:16.354472    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/42551b05-0c68-471b-883d-c7df49c9f2cd-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:17.037734 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:17.037706    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"
Oct 29 23:31:17.037902 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:17.037739    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z" event={"ID":"42551b05-0c68-471b-883d-c7df49c9f2cd","Type":"ContainerDied","Data":"abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9"}
Oct 29 23:31:17.037902 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:17.037812    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abfbe1f643f3de4af420b8064220e52ecb0f4200037f91524a99d4e4e3fc32d9"
Oct 29 23:31:17.090351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:17.090318    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:19.372334 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:19.372295    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:19.902437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:19.902398    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:20.755786 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:20.755749    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" containerID="cri-o://d41dbca38ab29946fe8c4e88df7df14fa94773487a5244e5da4e8e254128774a" gracePeriod=45
Oct 29 23:31:20.843853 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:20.843822    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": dial tcp 10.132.0.47:8000: connect: connection refused"
Oct 29 23:31:20.975859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:20.975799    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main" containerID="cri-o://51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f" gracePeriod=45
Oct 29 23:31:21.069171 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.069082    5111 generic.go:356] "Generic (PLEG): container finished" podID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerID="d41dbca38ab29946fe8c4e88df7df14fa94773487a5244e5da4e8e254128774a" exitCode=0
Oct 29 23:31:21.069308 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.069154    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" event={"ID":"0c9c00e5-b490-4a4c-9d18-a73a2250fb72","Type":"ContainerDied","Data":"d41dbca38ab29946fe8c4e88df7df14fa94773487a5244e5da4e8e254128774a"}
Oct 29 23:31:21.125863 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.125582    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:31:21.203834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.203804    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.204076 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204037    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.204186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204096    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.204186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204148    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.204434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204410    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.205493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204615    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\" (UniqueName: \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.205493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204669    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.205493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.204779    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\") pod \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\" (UID: \"0c9c00e5-b490-4a4c-9d18-a73a2250fb72\") "
Oct 29 23:31:21.205493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.205188    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.205493 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.205208    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.206198 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.206175    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.212277 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.212241    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:21.217690 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.217655    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "kube-api-access-zd4fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:21.218307 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.218273    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm" pod "0c9c00e5-b490-4a4c-9d18-a73a2250fb72" (UID: "0c9c00e5-b490-4a4c-9d18-a73a2250fb72"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.243263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.243190    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:21.277983 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.277940    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:31:21.305536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305516    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\" (UniqueName: \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305594    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305702 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305666    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305701    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\" (UniqueName: \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305741 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305730    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305839 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305763    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\") pod \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\" (UID: \"2dfd907e-011e-46b5-a4f7-a53e8531a8be\") "
Oct 29 23:31:21.305839 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305821    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.306010 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.305946    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.306192 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306166    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306189    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306209    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306223    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306235    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306248    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd4fz\" (UniqueName: \"kubernetes.io/projected/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-kube-api-access-zd4fz\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.306302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.306260    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c9c00e5-b490-4a4c-9d18-a73a2250fb72-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.308075 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.308055    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:21.308235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.308215    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:21.308304 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.308284    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk" pod "2dfd907e-011e-46b5-a4f7-a53e8531a8be" (UID: "2dfd907e-011e-46b5-a4f7-a53e8531a8be"). InnerVolumeSpecName "kube-api-access-m7szk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:21.406923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.406881    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7szk\" (UniqueName: \"kubernetes.io/projected/2dfd907e-011e-46b5-a4f7-a53e8531a8be-kube-api-access-m7szk\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.406923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.406918    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.406923 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.406931    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.407182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.406942    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2dfd907e-011e-46b5-a4f7-a53e8531a8be-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:21.658745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:21.658671    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:22.076697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.076664    5111 generic.go:356] "Generic (PLEG): container finished" podID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerID="51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f" exitCode=0
Oct 29 23:31:22.077122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.076698    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" event={"ID":"2dfd907e-011e-46b5-a4f7-a53e8531a8be","Type":"ContainerDied","Data":"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"}
Oct 29 23:31:22.077122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.076711    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"
Oct 29 23:31:22.077122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.076744    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448" event={"ID":"2dfd907e-011e-46b5-a4f7-a53e8531a8be","Type":"ContainerDied","Data":"3d5f3cfc66c2fa743a94e211bc226f19823f0d1142372e63333520ec3fa18de2"}
Oct 29 23:31:22.077122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.076770    5111 scope.go:122] "RemoveContainer" containerID="51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"
Oct 29 23:31:22.078254 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.078223    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"
Oct 29 23:31:22.078390 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.078251    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9" event={"ID":"0c9c00e5-b490-4a4c-9d18-a73a2250fb72","Type":"ContainerDied","Data":"5262ac56969a97bbcf1f46c4c75d1f362ddd94623607019f55f1960722c4ee05"}
Oct 29 23:31:22.088711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.088693    5111 scope.go:122] "RemoveContainer" containerID="51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"
Oct 29 23:31:22.089058 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:22.089033    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f\": container with ID starting with 51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f not found: ID does not exist" containerID="51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"
Oct 29 23:31:22.089152 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.089066    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f"} err="failed to get container status \"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f\": rpc error: code = NotFound desc = could not find container \"51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f\": container with ID starting with 51c7da8e33402315209613fdb37c3e86102a4eb8402a01f6214a2e4fb97fb52f not found: ID does not exist"
Oct 29 23:31:22.089152 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.089085    5111 scope.go:122] "RemoveContainer" containerID="d41dbca38ab29946fe8c4e88df7df14fa94773487a5244e5da4e8e254128774a"
Oct 29 23:31:22.133749 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.133725    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:31:22.147564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.147542    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/simulator-w2-be8726-kserve-6555f5746f-46lx9"]
Oct 29 23:31:22.174788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.174769    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:31:22.178080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:22.178060    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-be8726/premium-w2-be8726-kserve-7f9968956-ms448"]
Oct 29 23:31:23.000298 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.000272    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:23.061859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.061827    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" path="/var/lib/kubelet/pods/0c9c00e5-b490-4a4c-9d18-a73a2250fb72/volumes"
Oct 29 23:31:23.062305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.062290    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" path="/var/lib/kubelet/pods/2dfd907e-011e-46b5-a4f7-a53e8531a8be/volumes"
Oct 29 23:31:23.810683 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.810653    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"]
Oct 29 23:31:23.811120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811105    5111 cpu_manager.go:414] "containerMap: removing container" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main"
Oct 29 23:31:23.811162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811120    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main"
Oct 29 23:31:23.811162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811140    5111 cpu_manager.go:414] "containerMap: removing container" podUID="42551b05-0c68-471b-883d-c7df49c9f2cd" containerName="revoke-keys"
Oct 29 23:31:23.811162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811145    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42551b05-0c68-471b-883d-c7df49c9f2cd" containerName="revoke-keys"
Oct 29 23:31:23.811162 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811161    5111 cpu_manager.go:414] "containerMap: removing container" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main"
Oct 29 23:31:23.811289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811167    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main"
Oct 29 23:31:23.811289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811252    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="42551b05-0c68-471b-883d-c7df49c9f2cd" containerName="revoke-keys"
Oct 29 23:31:23.811289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811263    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="2dfd907e-011e-46b5-a4f7-a53e8531a8be" containerName="main"
Oct 29 23:31:23.811289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.811270    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c9c00e5-b490-4a4c-9d18-a73a2250fb72" containerName="main"
Oct 29 23:31:23.817344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.817319    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.832354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.832322    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.832469 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.832402    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.832604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.832570    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkv5k\" (UniqueName: \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.833615 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.833592    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"]
Oct 29 23:31:23.843643 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.843619    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:31:23.843745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.843707    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:31:23.843804 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.843750    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:31:23.848145 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.848126    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:31:23.933664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.933638    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.933745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.933693    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkv5k\" (UniqueName: \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.933879 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.933860    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.934458 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.934441    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.935942 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.935921    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:23.954774 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:23.954748    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkv5k\" (UniqueName: \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\") pod \"maas-api-revoke-sub-d6b0f1b8-dn4kk\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:24.133451 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:24.133419    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:24.202779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:24.202744    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:24.442681 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:24.442652    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"]
Oct 29 23:31:24.446278 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:31:24.446243    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991d10ac_9d46_41e1_a208_d1ca31a8d488.slice/crio-7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714 WatchSource:0}: Error finding container 7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714: Status 404 returned error can't find the container with id 7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714
Oct 29 23:31:24.767419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:24.767346    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:24.904810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:24.904776    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:25.039008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:25.038901    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:25.098034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:25.098001    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk" event={"ID":"991d10ac-9d46-41e1-a208-d1ca31a8d488","Type":"ContainerStarted","Data":"f74ed119fd0a81c3e04af3d23eed52dbba19d607374e13f924b44eb8c797e37b"}
Oct 29 23:31:25.098186 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:25.098053    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk" event={"ID":"991d10ac-9d46-41e1-a208-d1ca31a8d488","Type":"ContainerStarted","Data":"7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714"}
Oct 29 23:31:25.141649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:25.141607    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk" podStartSLOduration=2.141594573 podStartE2EDuration="2.141594573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:25.140208995 +0000 UTC m=+1096.226492129" watchObservedRunningTime="2026-10-29 23:31:25.141594573 +0000 UTC m=+1096.227877708"
Oct 29 23:31:25.444848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:25.444818    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:26.028608 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.028573    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"]
Oct 29 23:31:26.042042 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.041961    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-adf69316-7mr45"]
Oct 29 23:31:26.106313 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.106284    5111 generic.go:356] "Generic (PLEG): container finished" podID="991d10ac-9d46-41e1-a208-d1ca31a8d488" containerID="f74ed119fd0a81c3e04af3d23eed52dbba19d607374e13f924b44eb8c797e37b" exitCode=0
Oct 29 23:31:26.106440 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.106359    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk" event={"ID":"991d10ac-9d46-41e1-a208-d1ca31a8d488","Type":"ContainerDied","Data":"f74ed119fd0a81c3e04af3d23eed52dbba19d607374e13f924b44eb8c797e37b"}
Oct 29 23:31:26.147822 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:26.147789    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:31:26.469169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.469136    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:26.589139 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.589102    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:31:26.745051 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:26.744924    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:27.061167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.061096    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd0808ad-6aa3-4b8a-9370-590af9bff895" path="/var/lib/kubelet/pods/dd0808ad-6aa3-4b8a-9370-590af9bff895/volumes"
Oct 29 23:31:27.438468 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.438437    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:27.568533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.568499    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\" (UniqueName: \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\") pod \"991d10ac-9d46-41e1-a208-d1ca31a8d488\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") "
Oct 29 23:31:27.568793 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.568759    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\") pod \"991d10ac-9d46-41e1-a208-d1ca31a8d488\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") "
Oct 29 23:31:27.569027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.569006    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\") pod \"991d10ac-9d46-41e1-a208-d1ca31a8d488\" (UID: \"991d10ac-9d46-41e1-a208-d1ca31a8d488\") "
Oct 29 23:31:27.569952 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.569926    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca" pod "991d10ac-9d46-41e1-a208-d1ca31a8d488" (UID: "991d10ac-9d46-41e1-a208-d1ca31a8d488"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:27.587524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.585947    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca" pod "991d10ac-9d46-41e1-a208-d1ca31a8d488" (UID: "991d10ac-9d46-41e1-a208-d1ca31a8d488"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:27.588272 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.588238    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k" pod "991d10ac-9d46-41e1-a208-d1ca31a8d488" (UID: "991d10ac-9d46-41e1-a208-d1ca31a8d488"). InnerVolumeSpecName "kube-api-access-zkv5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:27.670679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.670608    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:27.670679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.670635    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkv5k\" (UniqueName: \"kubernetes.io/projected/991d10ac-9d46-41e1-a208-d1ca31a8d488-kube-api-access-zkv5k\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:27.670679 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:27.670646    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/991d10ac-9d46-41e1-a208-d1ca31a8d488-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:28.131111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:28.131071    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk" event={"ID":"991d10ac-9d46-41e1-a208-d1ca31a8d488","Type":"ContainerDied","Data":"7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714"}
Oct 29 23:31:28.131573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:28.131178    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b781658dfce36083355bd4618693b9df082cf2023403bc6906acda0e43eb714"
Oct 29 23:31:28.131573 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:28.131263    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"
Oct 29 23:31:28.201928 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:28.201896    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:29.607437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:29.607401    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:31.028036 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:31.028001    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:31.869596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:31.869564    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:32.011829 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.011793    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:32.175836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.174491    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:31:32.175836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.174719    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" containerID="cri-o://8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430" gracePeriod=30
Oct 29 23:31:32.187430 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.187404    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:31:32.187603 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.187583    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" containerID="cri-o://fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6" gracePeriod=30
Oct 29 23:31:32.200805 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.200778    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:31:32.200966 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.200947    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy" containerID="cri-o://50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740" gracePeriod=30
Oct 29 23:31:32.242306 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.242281    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:32.541522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.541451    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:32.612890 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.612853    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:31:32.613038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.612909    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:31:32.744828 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.744795    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:32.774408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.774375    5111 patch_prober.go:29] interesting pod/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 29 23:31:32.774570 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:32.774429    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 29 23:31:33.801234 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.801197    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"]
Oct 29 23:31:33.801715 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.801699    5111 cpu_manager.go:414] "containerMap: removing container" podUID="991d10ac-9d46-41e1-a208-d1ca31a8d488" containerName="revoke-keys"
Oct 29 23:31:33.801768 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.801717    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="991d10ac-9d46-41e1-a208-d1ca31a8d488" containerName="revoke-keys"
Oct 29 23:31:33.801811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.801796    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="991d10ac-9d46-41e1-a208-d1ca31a8d488" containerName="revoke-keys"
Oct 29 23:31:33.805563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.805538    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.828654 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.828616    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"]
Oct 29 23:31:33.829405 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.829379    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6n4b\" (UniqueName: \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.829535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.829476    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.829535 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.829500    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.901540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.901509    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:31:33.901540 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.901529    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:31:33.901713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.901574    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:31:33.901713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.901632    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:31:33.930193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.930170    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6n4b\" (UniqueName: \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.930297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.930257    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.930297 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.930288    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.930924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.930907    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.932559 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.932539    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:33.950861 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:33.950839    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6n4b\" (UniqueName: \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\") pod \"maas-api-revoke-sub-97784caf-bhnnh\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:34.119465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:34.119444    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:34.250044 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:34.250012    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:34.429328 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:34.429299    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"]
Oct 29 23:31:34.430250 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:31:34.430225    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf78e8e21_fb16_4eff_a071_b6b7f0b2908f.slice/crio-fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137 WatchSource:0}: Error finding container fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137: Status 404 returned error can't find the container with id fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137
Oct 29 23:31:34.850193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:34.850165    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:35.185924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:35.185888    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh" event={"ID":"f78e8e21-fb16-4eff-a071-b6b7f0b2908f","Type":"ContainerStarted","Data":"5738ac3bf4a904fdd7552c7dcca05e7d60753388262d9151b7204b9ca9707864"}
Oct 29 23:31:35.186079 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:35.185956    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh" event={"ID":"f78e8e21-fb16-4eff-a071-b6b7f0b2908f","Type":"ContainerStarted","Data":"fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137"}
Oct 29 23:31:35.229216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:35.229174    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh" podStartSLOduration=2.229160279 podStartE2EDuration="2.229160279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:35.228187937 +0000 UTC m=+1106.314471090" watchObservedRunningTime="2026-10-29 23:31:35.229160279 +0000 UTC m=+1106.315443413"
Oct 29 23:31:35.993513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:35.993472    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:36.028456 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:36.028425    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"]
Oct 29 23:31:36.041358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:36.041326    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-a8a15ef4-kcrdx"]
Oct 29 23:31:36.194382 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:36.194348    5111 generic.go:356] "Generic (PLEG): container finished" podID="f78e8e21-fb16-4eff-a071-b6b7f0b2908f" containerID="5738ac3bf4a904fdd7552c7dcca05e7d60753388262d9151b7204b9ca9707864" exitCode=0
Oct 29 23:31:36.194563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:36.194426    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh" event={"ID":"f78e8e21-fb16-4eff-a071-b6b7f0b2908f","Type":"ContainerDied","Data":"5738ac3bf4a904fdd7552c7dcca05e7d60753388262d9151b7204b9ca9707864"}
Oct 29 23:31:37.062375 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.062336    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7" path="/var/lib/kubelet/pods/6e3f91b6-1af9-4d67-9cbd-eb2a35dc44b7/volumes"
Oct 29 23:31:37.367395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.367171    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:37.466660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.466633    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\") pod \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") "
Oct 29 23:31:37.466843 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.466796    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\") pod \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") "
Oct 29 23:31:37.466910 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.466858    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\" (UniqueName: \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\") pod \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\" (UID: \"f78e8e21-fb16-4eff-a071-b6b7f0b2908f\") "
Oct 29 23:31:37.467467 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.467434    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca" pod "f78e8e21-fb16-4eff-a071-b6b7f0b2908f" (UID: "f78e8e21-fb16-4eff-a071-b6b7f0b2908f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:37.469745 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.469711    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca" pod "f78e8e21-fb16-4eff-a071-b6b7f0b2908f" (UID: "f78e8e21-fb16-4eff-a071-b6b7f0b2908f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:37.471285 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.470605    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b" pod "f78e8e21-fb16-4eff-a071-b6b7f0b2908f" (UID: "f78e8e21-fb16-4eff-a071-b6b7f0b2908f"). InnerVolumeSpecName "kube-api-access-z6n4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.475115 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.475092    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:31:37.514707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.514684    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:31:37.519761 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.519735    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:31:37.568416 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568378    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.568584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568430    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.568584 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568474    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.568710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568578    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.568710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568632    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.568710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568689    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.568873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568741    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.568873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568785    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.568873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568818    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.568873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568860    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568893    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568924    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.568955    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569017    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569050    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.569116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569078    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569132    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569140    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569139    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569214    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569328    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569345    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569369    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.569420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569409    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569468    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569506    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569550    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569551    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569606    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\") pod \"631fe478-a305-4be7-93e5-7759979632e5\" (UID: \"631fe478-a305-4be7-93e5-7759979632e5\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569614    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569651    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\") pod \"a8aeec2e-d846-4de9-a12b-f673207514e4\" (UID: \"a8aeec2e-d846-4de9-a12b-f673207514e4\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569689    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\") pod \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\" (UID: \"3ee79c4c-aa1a-4bd8-968d-3e8df0501592\") "
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569902    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.570032 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.569879    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570218    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570227    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570262    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570280    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570297    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6n4b\" (UniqueName: \"kubernetes.io/projected/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-kube-api-access-z6n4b\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570314    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570328    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/631fe478-a305-4be7-93e5-7759979632e5-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570343    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570356    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570370    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f78e8e21-fb16-4eff-a071-b6b7f0b2908f-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570385    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a8aeec2e-d846-4de9-a12b-f673207514e4-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.570520 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570276    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:37.571290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570622    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.571290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.570815    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.573474 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573442    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.573596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573483    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.573596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573484    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.573771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573600    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.573771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573618    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "kube-api-access-xdlhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.574149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573737    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "kube-api-access-zc6v4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.574149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.573839    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:37.574149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.574078    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.574149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.574108    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.574542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.574284    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket" pod "a8aeec2e-d846-4de9-a12b-f673207514e4" (UID: "a8aeec2e-d846-4de9-a12b-f673207514e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.574542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.574293    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.574542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.574341    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:37.575620 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.575593    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "kube-api-access-fhplr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.576013 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.575969    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo" pod "3ee79c4c-aa1a-4bd8-968d-3e8df0501592" (UID: "3ee79c4c-aa1a-4bd8-968d-3e8df0501592"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:37.576137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.576037    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:37.576362 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.576346    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo" pod "631fe478-a305-4be7-93e5-7759979632e5" (UID: "631fe478-a305-4be7-93e5-7759979632e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:37.658123 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.658086    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:37.671149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671111    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671144    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc6v4\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-kube-api-access-zc6v4\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671149 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671155    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671164    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671174    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/631fe478-a305-4be7-93e5-7759979632e5-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671182    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhplr\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-kube-api-access-fhplr\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671191    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671199    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/631fe478-a305-4be7-93e5-7759979632e5-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671207    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a8aeec2e-d846-4de9-a12b-f673207514e4-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671214    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671222    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671230    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdlhl\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-kube-api-access-xdlhl\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671237    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a8aeec2e-d846-4de9-a12b-f673207514e4-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671245    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671252    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671259    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/631fe478-a305-4be7-93e5-7759979632e5-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671267    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671274    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671282    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:37.671472 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:37.671289    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ee79c4c-aa1a-4bd8-968d-3e8df0501592-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:38.211420 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.211383    5111 generic.go:356] "Generic (PLEG): container finished" podID="631fe478-a305-4be7-93e5-7759979632e5" containerID="8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430" exitCode=0
Oct 29 23:31:38.211907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.211464    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"
Oct 29 23:31:38.211907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.211477    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" event={"ID":"631fe478-a305-4be7-93e5-7759979632e5","Type":"ContainerDied","Data":"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"}
Oct 29 23:31:38.211907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.211548    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n" event={"ID":"631fe478-a305-4be7-93e5-7759979632e5","Type":"ContainerDied","Data":"5e6c179efa5441973c2544ed96861be0bbe465a0a8862a00615b9a140de6550d"}
Oct 29 23:31:38.211907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.211581    5111 scope.go:122] "RemoveContainer" containerID="8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"
Oct 29 23:31:38.213249 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.213227    5111 generic.go:356] "Generic (PLEG): container finished" podID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerID="50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740" exitCode=0
Oct 29 23:31:38.213329 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.213297    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"
Oct 29 23:31:38.213390 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.213298    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" event={"ID":"3ee79c4c-aa1a-4bd8-968d-3e8df0501592","Type":"ContainerDied","Data":"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"}
Oct 29 23:31:38.213446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.213431    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn" event={"ID":"3ee79c4c-aa1a-4bd8-968d-3e8df0501592","Type":"ContainerDied","Data":"325eff6bf90010c11d20bd66063c7108e074367d9b85750ff5c13a49066e2689"}
Oct 29 23:31:38.215811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.215529    5111 generic.go:356] "Generic (PLEG): container finished" podID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerID="fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6" exitCode=0
Oct 29 23:31:38.215811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.215582    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" event={"ID":"a8aeec2e-d846-4de9-a12b-f673207514e4","Type":"ContainerDied","Data":"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"}
Oct 29 23:31:38.215811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.215603    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"
Oct 29 23:31:38.215811 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.215650    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm" event={"ID":"a8aeec2e-d846-4de9-a12b-f673207514e4","Type":"ContainerDied","Data":"e861d85b18b61a4e47d9066ed5c4b2343f6547eadb55803e1ea4b054a07be06c"}
Oct 29 23:31:38.219207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.219175    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh" event={"ID":"f78e8e21-fb16-4eff-a071-b6b7f0b2908f","Type":"ContainerDied","Data":"fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137"}
Oct 29 23:31:38.219322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.219224    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fdbce8bfd602b6ad80ea6afdb706e7a6e75c7bb5ef3c2f04b3d6da75f1c9d137"
Oct 29 23:31:38.219322 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.219260    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"
Oct 29 23:31:38.224619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.224604    5111 scope.go:122] "RemoveContainer" containerID="8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"
Oct 29 23:31:38.224949 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:38.224927    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430\": container with ID starting with 8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430 not found: ID does not exist" containerID="8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"
Oct 29 23:31:38.225095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.224961    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430"} err="failed to get container status \"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430\": rpc error: code = NotFound desc = could not find container \"8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430\": container with ID starting with 8aac7037da45b41adc5ad004a0fe0c88ae948eef254704e15a48d12e798fc430 not found: ID does not exist"
Oct 29 23:31:38.225095 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.224999    5111 scope.go:122] "RemoveContainer" containerID="50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"
Oct 29 23:31:38.234791 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.234771    5111 scope.go:122] "RemoveContainer" containerID="50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"
Oct 29 23:31:38.235115 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:38.235094    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740\": container with ID starting with 50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740 not found: ID does not exist" containerID="50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"
Oct 29 23:31:38.235182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.235123    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740"} err="failed to get container status \"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740\": rpc error: code = NotFound desc = could not find container \"50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740\": container with ID starting with 50a7154c6430ffaafec96e8627fcb683075dc79824f50c95afea6e272835d740 not found: ID does not exist"
Oct 29 23:31:38.235182 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.235140    5111 scope.go:122] "RemoveContainer" containerID="fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"
Oct 29 23:31:38.244070 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.244051    5111 scope.go:122] "RemoveContainer" containerID="fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"
Oct 29 23:31:38.244361 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:38.244330    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6\": container with ID starting with fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6 not found: ID does not exist" containerID="fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"
Oct 29 23:31:38.244434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.244371    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6"} err="failed to get container status \"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6\": rpc error: code = NotFound desc = could not find container \"fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6\": container with ID starting with fd8a3cec268182359df775b17ee5ecea0bfaf58fdd2b5ab7901c166f79e34cf6 not found: ID does not exist"
Oct 29 23:31:38.287428 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.287390    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:31:38.290950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.290926    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-d6n5n"]
Oct 29 23:31:38.343258 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.343218    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:31:38.349465 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.349436    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-xj4vm"]
Oct 29 23:31:38.402146 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.402112    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:31:38.406000 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:38.405951    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-be8726-openshift-default-55dcd5768f-lz8bn"]
Oct 29 23:31:39.032289 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.032259    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"]
Oct 29 23:31:39.035613 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.035582    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f506b116-7xvzk"]
Oct 29 23:31:39.062086 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.062054    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" path="/var/lib/kubelet/pods/3ee79c4c-aa1a-4bd8-968d-3e8df0501592/volumes"
Oct 29 23:31:39.062547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.062529    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="631fe478-a305-4be7-93e5-7759979632e5" path="/var/lib/kubelet/pods/631fe478-a305-4be7-93e5-7759979632e5/volumes"
Oct 29 23:31:39.062899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.062887    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66e4932e-c3a5-4d60-8194-9c7ea47fefb3" path="/var/lib/kubelet/pods/66e4932e-c3a5-4d60-8194-9c7ea47fefb3/volumes"
Oct 29 23:31:39.063612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.063595    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" path="/var/lib/kubelet/pods/a8aeec2e-d846-4de9-a12b-f673207514e4/volumes"
Oct 29 23:31:39.093783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.093749    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:39.766392 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.766356    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:31:39.781860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.781830    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:31:39.883858 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:39.883827    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:40.576568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:40.576534    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:41.609228 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:41.609184    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main" containerID="cri-o://4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06" gracePeriod=45
Oct 29 23:31:41.882265 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:41.882232    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:31:42.023671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.023616    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.023937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.023729    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.023937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.023762    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.023937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.023824    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.024200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.023956    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\" (UniqueName: \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.024200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.024089    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\") pod \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\" (UID: \"b4edd79f-1a7e-420b-a1b3-4ebd714f3352\") "
Oct 29 23:31:42.024711 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.024675    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:42.025130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.025101    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:42.025227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.025176    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:42.031061 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.031009    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "kube-api-access-5j9l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:42.032165 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.032119    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:42.034196 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.034145    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs" pod "b4edd79f-1a7e-420b-a1b3-4ebd714f3352" (UID: "b4edd79f-1a7e-420b-a1b3-4ebd714f3352"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:42.056907 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.056872    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125141    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9l5\" (UniqueName: \"kubernetes.io/projected/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-kube-api-access-5j9l5\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125177    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125188    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125218    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125227    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.125236 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.125236    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4edd79f-1a7e-420b-a1b3-4ebd714f3352-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:42.192137 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.192091    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:42.248072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.248040    5111 generic.go:356] "Generic (PLEG): container finished" podID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerID="4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06" exitCode=0
Oct 29 23:31:42.248270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.248081    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"
Oct 29 23:31:42.248270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.248120    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" event={"ID":"b4edd79f-1a7e-420b-a1b3-4ebd714f3352","Type":"ContainerDied","Data":"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"}
Oct 29 23:31:42.248270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.248170    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86" event={"ID":"b4edd79f-1a7e-420b-a1b3-4ebd714f3352","Type":"ContainerDied","Data":"96b40ed3fee34276d698bcbd0690d90746ade5e7360bf2464a17f88e9c4db88a"}
Oct 29 23:31:42.248270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.248191    5111 scope.go:122] "RemoveContainer" containerID="4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"
Oct 29 23:31:42.260058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.260041    5111 scope.go:122] "RemoveContainer" containerID="4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"
Oct 29 23:31:42.260362 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:42.260344    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06\": container with ID starting with 4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06 not found: ID does not exist" containerID="4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"
Oct 29 23:31:42.260425 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.260377    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06"} err="failed to get container status \"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06\": rpc error: code = NotFound desc = could not find container \"4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06\": container with ID starting with 4a0cd44cd1aa1a49770cd6528a2cf86f825fbb105fe10e52173c4c738c08df06 not found: ID does not exist"
Oct 29 23:31:42.311645 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.311613    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:31:42.314757 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.314724    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-9280e4/simulator-w3-9280e4-kserve-c8fc9cd48-46s86"]
Oct 29 23:31:42.398496 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.398398    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:42.695553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.695459    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:42.870439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:42.870403    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:43.062048 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:43.061924    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" path="/var/lib/kubelet/pods/b4edd79f-1a7e-420b-a1b3-4ebd714f3352/volumes"
Oct 29 23:31:43.553471 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:43.553407    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:44.329870 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:44.329834    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:44.991827 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:44.991791    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:45.687415 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:45.687385    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:45.840530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:45.840491    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:46.408927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:46.408890    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:46.880437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:46.880393    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:47.027695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:47.027316    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:47.503899 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:47.503853    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:48.322701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.322668    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"]
Oct 29 23:31:48.323303 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323282    5111 cpu_manager.go:414] "containerMap: removing container" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323305    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323314    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323319    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323333    5111 cpu_manager.go:414] "containerMap: removing container" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323338    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy"
Oct 29 23:31:48.323346 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323345    5111 cpu_manager.go:414] "containerMap: removing container" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323350    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323365    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f78e8e21-fb16-4eff-a071-b6b7f0b2908f" containerName="revoke-keys"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323369    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78e8e21-fb16-4eff-a071-b6b7f0b2908f" containerName="revoke-keys"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323442    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4edd79f-1a7e-420b-a1b3-4ebd714f3352" containerName="main"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323451    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f78e8e21-fb16-4eff-a071-b6b7f0b2908f" containerName="revoke-keys"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323459    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="631fe478-a305-4be7-93e5-7759979632e5" containerName="istio-proxy"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323465    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a8aeec2e-d846-4de9-a12b-f673207514e4" containerName="istio-proxy"
Oct 29 23:31:48.323533 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.323473    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ee79c4c-aa1a-4bd8-968d-3e8df0501592" containerName="istio-proxy"
Oct 29 23:31:48.328437 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.328415    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.348308 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.348278    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"]
Oct 29 23:31:48.409194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.409160    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.409374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.409221    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.409374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.409336    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twt8s\" (UniqueName: \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.422582 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.422553    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:31:48.422753 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.422651    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:31:48.422832 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.422817    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:31:48.422924 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.422912    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:31:48.509939 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.509902    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.510153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.509955    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twt8s\" (UniqueName: \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.510153 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.510089    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.510738 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.510717    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.512422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.512399    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.530368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.530344    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twt8s\" (UniqueName: \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\") pod \"maas-api-revoke-sub-1a8d697f-bflld\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.644481 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.644446    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:48.945693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:48.945659    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"]
Oct 29 23:31:48.946991 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:31:48.946948    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod145ea915_7b9c_4279_8f1f_705a1033cb0c.slice/crio-4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb WatchSource:0}: Error finding container 4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb: Status 404 returned error can't find the container with id 4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb
Oct 29 23:31:49.299640 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:49.299601    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld" event={"ID":"145ea915-7b9c-4279-8f1f-705a1033cb0c","Type":"ContainerStarted","Data":"474c922eb4de27d7b98252f47091bd98ed6e7f09a91376652a7ed50df7fc74b1"}
Oct 29 23:31:49.299788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:49.299670    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld" event={"ID":"145ea915-7b9c-4279-8f1f-705a1033cb0c","Type":"ContainerStarted","Data":"4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb"}
Oct 29 23:31:49.342031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:49.341964    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld" podStartSLOduration=1.341950771 podStartE2EDuration="1.341950771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:49.340137723 +0000 UTC m=+1120.426420861" watchObservedRunningTime="2026-10-29 23:31:49.341950771 +0000 UTC m=+1120.428233904"
Oct 29 23:31:50.033524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:50.033488    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"]
Oct 29 23:31:50.046783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:50.046754    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-cfa936f8-hrf8n"]
Oct 29 23:31:50.107091 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:50.107054    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:50.307935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:50.307850    5111 generic.go:356] "Generic (PLEG): container finished" podID="145ea915-7b9c-4279-8f1f-705a1033cb0c" containerID="474c922eb4de27d7b98252f47091bd98ed6e7f09a91376652a7ed50df7fc74b1" exitCode=0
Oct 29 23:31:50.307935 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:50.307892    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld" event={"ID":"145ea915-7b9c-4279-8f1f-705a1033cb0c","Type":"ContainerDied","Data":"474c922eb4de27d7b98252f47091bd98ed6e7f09a91376652a7ed50df7fc74b1"}
Oct 29 23:31:51.061275 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.061244    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6785f52-e79c-42ee-88c7-62c15035a994" path="/var/lib/kubelet/pods/a6785f52-e79c-42ee-88c7-62c15035a994/volumes"
Oct 29 23:31:51.458635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.458611    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:51.547323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.547281    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\") pod \"145ea915-7b9c-4279-8f1f-705a1033cb0c\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") "
Oct 29 23:31:51.547505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.547385    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\") pod \"145ea915-7b9c-4279-8f1f-705a1033cb0c\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") "
Oct 29 23:31:51.547505 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.547412    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\" (UniqueName: \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\") pod \"145ea915-7b9c-4279-8f1f-705a1033cb0c\" (UID: \"145ea915-7b9c-4279-8f1f-705a1033cb0c\") "
Oct 29 23:31:51.547713 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.547686    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca" pod "145ea915-7b9c-4279-8f1f-705a1033cb0c" (UID: "145ea915-7b9c-4279-8f1f-705a1033cb0c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:51.549482 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.549454    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca" pod "145ea915-7b9c-4279-8f1f-705a1033cb0c" (UID: "145ea915-7b9c-4279-8f1f-705a1033cb0c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:51.549579 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.549523    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s" pod "145ea915-7b9c-4279-8f1f-705a1033cb0c" (UID: "145ea915-7b9c-4279-8f1f-705a1033cb0c"). InnerVolumeSpecName "kube-api-access-twt8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:51.648990 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.648954    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:51.649147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.649007    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twt8s\" (UniqueName: \"kubernetes.io/projected/145ea915-7b9c-4279-8f1f-705a1033cb0c-kube-api-access-twt8s\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:51.649147 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:51.649019    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/145ea915-7b9c-4279-8f1f-705a1033cb0c-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:52.320810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:52.320774    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld" event={"ID":"145ea915-7b9c-4279-8f1f-705a1033cb0c","Type":"ContainerDied","Data":"4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb"}
Oct 29 23:31:52.320810 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:52.320810    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"
Oct 29 23:31:52.321313 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:52.320825    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4699cd9de2242bdc8a2916d1f8eb2014fbd8331e8c78aa1206bcc45a3dcb76bb"
Oct 29 23:31:53.664253 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:53.664216    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:53.880038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:53.880006    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:53.995290 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:53.995200    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:31:53.995490 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:53.995463    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy" containerID="cri-o://2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7" gracePeriod=30
Oct 29 23:31:54.008889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:54.008856    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:31:54.009103 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:54.009065    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy" containerID="cri-o://be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b" gracePeriod=30
Oct 29 23:31:54.111891 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:54.111860    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:31:54.790316 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:54.790268    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main" containerID="cri-o://877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad" gracePeriod=45
Oct 29 23:31:54.802434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:54.802400    5111 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main" containerID="cri-o://a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e" gracePeriod=45
Oct 29 23:31:55.049867 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.049833    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:31:55.076598 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.076573    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:31:55.187050 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187016    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187062    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\" (UniqueName: \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187084    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\" (UniqueName: \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187104    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187140    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187240 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187198    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\" (UniqueName: \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187240    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187298    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187341    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187352    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187381    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187422    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\") pod \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\" (UID: \"a0cd2e02-af0d-4ed4-922f-da7fac0a3726\") "
Oct 29 23:31:55.187501 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187485    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\") pod \"f3300e93-7cf1-407e-b46b-b43637553b86\" (UID: \"f3300e93-7cf1-407e-b46b-b43637553b86\") "
Oct 29 23:31:55.187872 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.187861    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.188078 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.188040    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.188188 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.188083    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.188293 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.188271    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.188358 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.188290    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.188516 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.188493    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.189730 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.189704    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.189873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.189791    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:55.189873 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.189824    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:31:55.190274 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.190255    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97" pod "f3300e93-7cf1-407e-b46b-b43637553b86" (UID: "f3300e93-7cf1-407e-b46b-b43637553b86"). InnerVolumeSpecName "kube-api-access-t4z97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:55.190422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.190404    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "kube-api-access-cpvb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:55.190460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.190417    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm" pod "a0cd2e02-af0d-4ed4-922f-da7fac0a3726" (UID: "a0cd2e02-af0d-4ed4-922f-da7fac0a3726"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:55.288611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288576    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288605    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpvb8\" (UniqueName: \"kubernetes.io/projected/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-kube-api-access-cpvb8\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288611 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288617    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4z97\" (UniqueName: \"kubernetes.io/projected/f3300e93-7cf1-407e-b46b-b43637553b86-kube-api-access-t4z97\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288626    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288636    5111 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-dshm\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288644    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3300e93-7cf1-407e-b46b-b43637553b86-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288653    5111 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tls-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288660    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288667    5111 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f3300e93-7cf1-407e-b46b-b43637553b86-home\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288676    5111 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-tmp-dir\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.288860 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.288683    5111 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0cd2e02-af0d-4ed4-922f-da7fac0a3726-model-cache\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:55.339359 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.339325    5111 generic.go:356] "Generic (PLEG): container finished" podID="f3300e93-7cf1-407e-b46b-b43637553b86" containerID="a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e" exitCode=0
Oct 29 23:31:55.339557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.339367    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"
Oct 29 23:31:55.339557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.339408    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" event={"ID":"f3300e93-7cf1-407e-b46b-b43637553b86","Type":"ContainerDied","Data":"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"}
Oct 29 23:31:55.339557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.339458    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx" event={"ID":"f3300e93-7cf1-407e-b46b-b43637553b86","Type":"ContainerDied","Data":"5b3dbf09838592083c114150608e0d9fc03b28a5907b32156bfc4333e31b5628"}
Oct 29 23:31:55.339557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.339478    5111 scope.go:122] "RemoveContainer" containerID="a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"
Oct 29 23:31:55.341263 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.341237    5111 generic.go:356] "Generic (PLEG): container finished" podID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerID="877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad" exitCode=0
Oct 29 23:31:55.341373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.341275    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" event={"ID":"a0cd2e02-af0d-4ed4-922f-da7fac0a3726","Type":"ContainerDied","Data":"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"}
Oct 29 23:31:55.341373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.341306    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s" event={"ID":"a0cd2e02-af0d-4ed4-922f-da7fac0a3726","Type":"ContainerDied","Data":"2880f7f511a701bb6980d351e41b0c017473cd83afc9bce478142c2bc164fcd4"}
Oct 29 23:31:55.341373 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.341279    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"
Oct 29 23:31:55.349805 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.349637    5111 scope.go:122] "RemoveContainer" containerID="a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"
Oct 29 23:31:55.349961 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:55.349941    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e\": container with ID starting with a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e not found: ID does not exist" containerID="a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"
Oct 29 23:31:55.350043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.350004    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e"} err="failed to get container status \"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e\": rpc error: code = NotFound desc = could not find container \"a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e\": container with ID starting with a9c0cf6e29d6ec8dc7da96b81ecef29a1b33970d70af0c48a12d16861baed34e not found: ID does not exist"
Oct 29 23:31:55.350043 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.350029    5111 scope.go:122] "RemoveContainer" containerID="877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"
Oct 29 23:31:55.358911 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.358885    5111 scope.go:122] "RemoveContainer" containerID="877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"
Oct 29 23:31:55.359183 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:55.359165    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad\": container with ID starting with 877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad not found: ID does not exist" containerID="877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"
Oct 29 23:31:55.359227 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.359190    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad"} err="failed to get container status \"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad\": rpc error: code = NotFound desc = could not find container \"877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad\": container with ID starting with 877f0ea1a9d35039d207dd24ce0770a953f55f76a840c8695e8cb8e692c050ad not found: ID does not exist"
Oct 29 23:31:55.395112 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.395067    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:31:55.397632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.397605    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/premium-w5-0b2ae9-kserve-69579cd9bb-nwd4s"]
Oct 29 23:31:55.436311 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.436244    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:31:55.439878 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:55.439848    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-0b2ae9/simulator-w5-0b2ae9-kserve-86d4c8fc75-ddlbx"]
Oct 29 23:31:56.148442 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:56.148409    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:31:57.061238 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:57.061203    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" path="/var/lib/kubelet/pods/a0cd2e02-af0d-4ed4-922f-da7fac0a3726/volumes"
Oct 29 23:31:57.061771 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:57.061749    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" path="/var/lib/kubelet/pods/f3300e93-7cf1-407e-b46b-b43637553b86/volumes"
Oct 29 23:31:59.274746 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.274685    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:31:59.291687 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.291668    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:31:59.321739 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321709    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.321859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321759    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.321859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321800    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.321859 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321830    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321865    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321896    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.321933    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322033    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322063    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322099    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322117    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322140    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322163    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322192    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322222    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322263    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322312    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322343    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322372    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\") pod \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\" (UID: \"00c6ed7b-e070-4cd0-bfd3-949afca12fa4\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322404    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322434    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\") pod \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\" (UID: \"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c\") "
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322453    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:59.322507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322470    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322725    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322743    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322781    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322801    5111 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istiod-ca-cert\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322815    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322829    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322841    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.323022 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.322931    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.323433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.323070    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.323433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.323136    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.324610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.324570    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:59.324843 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.324817    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.325298 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325264    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "kube-api-access-h629d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:59.325397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325328    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "kube-api-access-7lxgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:59.325461 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325433    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 29 23:31:59.325714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325696    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:59.325840 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325820    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token" pod "00c6ed7b-e070-4cd0-bfd3-949afca12fa4" (UID: "00c6ed7b-e070-4cd0-bfd3-949afca12fa4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:31:59.325908 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.325884    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy" pod "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" (UID: "36d1780e-f31a-4f2f-a0f1-ccd0d168f50c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 29 23:31:59.373951 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.373922    5111 generic.go:356] "Generic (PLEG): container finished" podID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerID="be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b" exitCode=0
Oct 29 23:31:59.374131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.373967    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"
Oct 29 23:31:59.374131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.374009    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" event={"ID":"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c","Type":"ContainerDied","Data":"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"}
Oct 29 23:31:59.374131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.374071    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8" event={"ID":"36d1780e-f31a-4f2f-a0f1-ccd0d168f50c","Type":"ContainerDied","Data":"8cc787db11faecde14ac52fd34f27b198847044c16bb13e0aea7d29079094c38"}
Oct 29 23:31:59.374131 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.374103    5111 scope.go:122] "RemoveContainer" containerID="be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"
Oct 29 23:31:59.377938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.377887    5111 generic.go:356] "Generic (PLEG): container finished" podID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerID="2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7" exitCode=0
Oct 29 23:31:59.377938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.377921    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"
Oct 29 23:31:59.377938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.377923    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" event={"ID":"00c6ed7b-e070-4cd0-bfd3-949afca12fa4","Type":"ContainerDied","Data":"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"}
Oct 29 23:31:59.378117 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.377996    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz" event={"ID":"00c6ed7b-e070-4cd0-bfd3-949afca12fa4","Type":"ContainerDied","Data":"5217a9f1b741e49468cdb88cb9ee0cda50a4a25963d6a79f462e0fd22b9fff0f"}
Oct 29 23:31:59.388209 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.388099    5111 scope.go:122] "RemoveContainer" containerID="be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"
Oct 29 23:31:59.388538 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:59.388509    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b\": container with ID starting with be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b not found: ID does not exist" containerID="be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"
Oct 29 23:31:59.388678 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.388548    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b"} err="failed to get container status \"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b\": rpc error: code = NotFound desc = could not find container \"be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b\": container with ID starting with be662250595ab94ac2d036d7f04391bdd71db1b800b284d4994b3b89adc01c3b not found: ID does not exist"
Oct 29 23:31:59.388678 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.388573    5111 scope.go:122] "RemoveContainer" containerID="2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"
Oct 29 23:31:59.398196 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.398181    5111 scope.go:122] "RemoveContainer" containerID="2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"
Oct 29 23:31:59.398472 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:31:59.398455    5111 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7\": container with ID starting with 2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7 not found: ID does not exist" containerID="2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"
Oct 29 23:31:59.398526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.398480    5111 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7"} err="failed to get container status \"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7\": rpc error: code = NotFound desc = could not find container \"2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7\": container with ID starting with 2cf3dc18d334c58c14e39a2dc6ebe6194a333ec68d40272eeb40645565b5b9f7 not found: ID does not exist"
Oct 29 23:31:59.424259 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424237    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424260    5111 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-certs\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424273    5111 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-credential-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424286    5111 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-workload-socket\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424299    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424311    5111 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-data\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424323    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424335    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lxgp\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-kube-api-access-7lxgp\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424348    5111 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-podinfo\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424369 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424362    5111 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-token\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424375    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424387    5111 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-istio-envoy\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.424648 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.424400    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h629d\" (UniqueName: \"kubernetes.io/projected/00c6ed7b-e070-4cd0-bfd3-949afca12fa4-kube-api-access-h629d\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:31:59.434848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.434821    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:31:59.438485 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.438463    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-6hsr8"]
Oct 29 23:31:59.475802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.475777    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:31:59.478021 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:31:59.478000    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-9280e4-openshift-default-6c769b66b7-lktlz"]
Oct 29 23:32:00.362905 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:00.362869    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:01.060798 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:01.060751    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" path="/var/lib/kubelet/pods/00c6ed7b-e070-4cd0-bfd3-949afca12fa4/volumes"
Oct 29 23:32:01.061218 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:01.061204    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" path="/var/lib/kubelet/pods/36d1780e-f31a-4f2f-a0f1-ccd0d168f50c/volumes"
Oct 29 23:32:03.226348 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:03.226281    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:03.437958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:03.437914    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:03.580460 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:03.580431    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:03.960326 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:03.960244    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:04.111216 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:04.111178    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:04.609985 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:04.609950    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:04.780412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:04.780361    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:10.784132 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:10.784090    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:11.018299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:11.018265    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:11.178124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:11.178090    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:11.340815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:11.340774    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:16.336193 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:16.336161    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:16.487947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:16.487913    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:17.099008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:17.098313    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:22.662352 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:22.662318    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:26.147718 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:32:26.147683    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:32:50.299397 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:50.299365    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:52.611835 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:32:52.611791    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:32:56.147883 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:32:56.147840    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:33:00.407877 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:00.407843    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:33:09.185632 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:09.185599    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:33:09.189887 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:09.189867    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:33:10.135536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:10.135504    5111 scope.go:122] "RemoveContainer" containerID="877297882d3ba39ceef0ab2f86f50b4e6b1e532d8fd2e538b8ca48aa4078078f"
Oct 29 23:33:10.147109 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:10.145125    5111 scope.go:122] "RemoveContainer" containerID="2bc1e4d3c8af9c72d6a68918ee8098d3a79335631c532c7b751a547c6a5f5d61"
Oct 29 23:33:10.155418 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:10.155393    5111 scope.go:122] "RemoveContainer" containerID="dfb777e597187c275b67aa975f164ae4e10af9961f6872934b262e0aa5843078"
Oct 29 23:33:10.163534 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:10.163513    5111 scope.go:122] "RemoveContainer" containerID="a05e58fa55a3416882aeb6b741e4f1e53e405ee6a63175f210f3de885143b516"
Oct 29 23:33:26.147586 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:33:26.147505    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:33:37.030366 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:37.030327    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"]
Oct 29 23:33:37.043833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:37.043807    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2114c313-r4ksf"]
Oct 29 23:33:37.060664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:37.060633    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79512ae5-129d-474a-a1a6-cc575480c631" path="/var/lib/kubelet/pods/79512ae5-129d-474a-a1a6-cc575480c631/volumes"
Oct 29 23:33:50.028354 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:50.028318    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"]
Oct 29 23:33:50.042011 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:50.041969    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-af510c64-c9mlb"]
Oct 29 23:33:51.060099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:51.060059    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a367053e-5414-4b97-8e0b-cf22e835ab14" path="/var/lib/kubelet/pods/a367053e-5414-4b97-8e0b-cf22e835ab14/volumes"
Oct 29 23:33:54.026207 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:54.026168    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"]
Oct 29 23:33:54.039522 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:54.039492    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-44903331-vn4x2"]
Oct 29 23:33:55.066100 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:55.066067    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28b88976-0b31-49c3-865e-4ef71d883307" path="/var/lib/kubelet/pods/28b88976-0b31-49c3-865e-4ef71d883307/volumes"
Oct 29 23:33:55.629164 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:33:55.629129    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:33:56.147214 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:33:56.147181    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:34:00.029144 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:00.029107    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"]
Oct 29 23:34:00.043428 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:00.043387    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-75099af1-hdtfr"]
Oct 29 23:34:00.519302 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:00.519269    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:34:01.062408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:01.062372    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="624b2813-eb9a-47e6-8ba5-e8f00d2c35f7" path="/var/lib/kubelet/pods/624b2813-eb9a-47e6-8ba5-e8f00d2c35f7/volumes"
Oct 29 23:34:26.147469 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:34:26.147429    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:34:27.029442 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:27.029403    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"]
Oct 29 23:34:27.042406 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:27.042376    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-975f1321-9cc66"]
Oct 29 23:34:27.060539 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:27.060510    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4adbb6cb-4b92-4430-983b-40f64cce4ecc" path="/var/lib/kubelet/pods/4adbb6cb-4b92-4430-983b-40f64cce4ecc/volumes"
Oct 29 23:34:30.026417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:30.026379    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"]
Oct 29 23:34:30.039553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:30.039521    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b47fbb16-f28ms"]
Oct 29 23:34:31.060587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:31.060555    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae70ecd5-bbde-4d76-b89e-f955b1bc3abb" path="/var/lib/kubelet/pods/ae70ecd5-bbde-4d76-b89e-f955b1bc3abb/volumes"
Oct 29 23:34:33.896671 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:33.896629    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:34:34.031403 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:34.031364    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:34:45.031194 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:45.031160    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"]
Oct 29 23:34:45.044487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:45.044463    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6566c8f5-zgwkp"]
Oct 29 23:34:45.060214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:45.060180    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5088088-4a8f-4ef0-9af9-65325f8b03c3" path="/var/lib/kubelet/pods/f5088088-4a8f-4ef0-9af9-65325f8b03c3/volumes"
Oct 29 23:34:46.033624 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:46.033593    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"]
Oct 29 23:34:46.047014 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:46.046986    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-663037f8-bnlt8"]
Oct 29 23:34:47.060393 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:47.060360    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad92f618-7f98-4939-80da-3e649335cc67" path="/var/lib/kubelet/pods/ad92f618-7f98-4939-80da-3e649335cc67/volumes"
Oct 29 23:34:52.027379 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:52.027303    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"]
Oct 29 23:34:52.040656 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:52.040630    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-114fa85f-m59j4"]
Oct 29 23:34:53.060580 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:53.060547    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="476ad2fc-49fb-4f64-ac9a-e18f154d2dcc" path="/var/lib/kubelet/pods/476ad2fc-49fb-4f64-ac9a-e18f154d2dcc/volumes"
Oct 29 23:34:56.147651 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:34:56.147616    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:34:59.026693 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:59.026660    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"]
Oct 29 23:34:59.040073 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:59.040046    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b1870970-r42vm"]
Oct 29 23:34:59.060917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:34:59.060890    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e555453-98e5-4aa4-97da-2a647289beba" path="/var/lib/kubelet/pods/1e555453-98e5-4aa4-97da-2a647289beba/volumes"
Oct 29 23:35:00.606743 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:00.606701    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:35:01.027271 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:01.027192    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"]
Oct 29 23:35:01.040424 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:01.040395    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5f82696-fvfd4"]
Oct 29 23:35:01.060337 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:01.060312    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8bd97219-5c5b-4417-943b-072cca4466ae" path="/var/lib/kubelet/pods/8bd97219-5c5b-4417-943b-072cca4466ae/volumes"
Oct 29 23:35:10.204138 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:10.204104    5111 scope.go:122] "RemoveContainer" containerID="5035e621cf52c62e96d479bbbf432cc04076e7d82e1252bf1a5ed44e7cfe9351"
Oct 29 23:35:10.213377 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:10.213346    5111 scope.go:122] "RemoveContainer" containerID="1f424d2675e7ef18755d5f10ae38c660be6772d05f2927d5488666b99c705732"
Oct 29 23:35:10.224833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:10.224810    5111 scope.go:122] "RemoveContainer" containerID="e9a19d04153865ec803168a30086850754d8a70e5ce35cabeb3211cd605228d7"
Oct 29 23:35:10.233024 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:10.233006    5111 scope.go:122] "RemoveContainer" containerID="e0d550840b513cf21faa1c6c67a989cbe686b2c3c22334c9ea56801ccfdc7044"
Oct 29 23:35:12.028513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:12.028473    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"]
Oct 29 23:35:12.042886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:12.042853    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f60470af-5rc4k"]
Oct 29 23:35:13.060317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:13.060287    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32d245a8-744e-4a7e-aaee-fb000959f749" path="/var/lib/kubelet/pods/32d245a8-744e-4a7e-aaee-fb000959f749/volumes"
Oct 29 23:35:16.029635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:16.029597    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"]
Oct 29 23:35:16.040917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:16.040888    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e9e2de7d-znmld"]
Oct 29 23:35:17.060792 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:17.060753    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790" path="/var/lib/kubelet/pods/ca9ef1d3-5e99-4c9e-83bf-08d6ca06d790/volumes"
Oct 29 23:35:18.027434 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:18.027401    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"]
Oct 29 23:35:18.040649 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:18.040612    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-bdb9f8fd-l9tmf"]
Oct 29 23:35:19.062426 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:19.062384    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f87007b6-23f4-402a-9459-62dfb6254f7e" path="/var/lib/kubelet/pods/f87007b6-23f4-402a-9459-62dfb6254f7e/volumes"
Oct 29 23:35:26.149334 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:35:26.149294    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:35:28.092896 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:28.092855    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:35:30.028295 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:30.028256    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"]
Oct 29 23:35:30.041616 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:30.041584    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-e3049769-qn7j8"]
Oct 29 23:35:31.028732 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:31.028690    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"]
Oct 29 23:35:31.042547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:31.042513    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-dae7dd00-dtm2w"]
Oct 29 23:35:31.060650 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:31.060617    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c2f2eed-a323-42ab-abd4-81d2e24483ed" path="/var/lib/kubelet/pods/1c2f2eed-a323-42ab-abd4-81d2e24483ed/volumes"
Oct 29 23:35:31.061057 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:31.061043    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ed8c2c5-f561-4d5a-9e19-d7204c2e855c" path="/var/lib/kubelet/pods/7ed8c2c5-f561-4d5a-9e19-d7204c2e855c/volumes"
Oct 29 23:35:43.537856 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:43.537821    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:35:43.670169 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:35:43.670143    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:35:56.169648 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:35:56.169616    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:36:00.717794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:00.717755    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:36:10.266261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.266223    5111 scope.go:122] "RemoveContainer" containerID="c052fdab00780833d8a4eb49ed4abbc3fd0f87a17ed9063e9ed7336169e86f70"
Oct 29 23:36:10.276279 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.276253    5111 scope.go:122] "RemoveContainer" containerID="5e29eed8c53268cb32cb3cf206f347fdf5680f2f6fc45445fdcdabbc3c354a3c"
Oct 29 23:36:10.285179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.285155    5111 scope.go:122] "RemoveContainer" containerID="64ece6ae73c83dc8a5aa4487f91a1f328f1b77eb711f430e4b8c1f45fd5b10f3"
Oct 29 23:36:10.293474 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.293454    5111 scope.go:122] "RemoveContainer" containerID="6164f3289dbf222ffd247644827b890367cd780fcf14cdade804af1a73fd8845"
Oct 29 23:36:10.301596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.301555    5111 scope.go:122] "RemoveContainer" containerID="6c5d0cb3f928ce31785ffc121364703f2167733740ea8ddea0b8444c958699b2"
Oct 29 23:36:10.309766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.309744    5111 scope.go:122] "RemoveContainer" containerID="a94985f74849d8bdc7210ca8ebccffd547d5c743c08559d02d52cd9e7c54438c"
Oct 29 23:36:10.317374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.317352    5111 scope.go:122] "RemoveContainer" containerID="fba922a25803f3a17241128cd927746b7ac5630a423f4893d5386c08b9270007"
Oct 29 23:36:10.324747 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.324724    5111 scope.go:122] "RemoveContainer" containerID="d820e5ae25c3523f3aba125a03df6a32384e8777bcf916370751063a41cb5033"
Oct 29 23:36:10.334829 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.334805    5111 scope.go:122] "RemoveContainer" containerID="563a58429e0b028eb629a72d7124d9a0fca1ad772cb1b8029ad5b19b5533d46d"
Oct 29 23:36:10.342134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:10.342118    5111 scope.go:122] "RemoveContainer" containerID="66d1a82f34210351304cf809a1ffb495d2e3c1aaa31c681a444712fc5ae74680"
Oct 29 23:36:17.030200 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:17.030161    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"]
Oct 29 23:36:17.043575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:17.043542    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-26c2e2a2-9vw8z"]
Oct 29 23:36:17.063155 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:17.063128    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42551b05-0c68-471b-883d-c7df49c9f2cd" path="/var/lib/kubelet/pods/42551b05-0c68-471b-883d-c7df49c9f2cd/volumes"
Oct 29 23:36:26.168157 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:36:26.168076    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:36:28.028944 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:28.028912    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"]
Oct 29 23:36:28.042513 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:28.042485    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-d6b0f1b8-dn4kk"]
Oct 29 23:36:29.061220 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:29.061183    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="991d10ac-9d46-41e1-a208-d1ca31a8d488" path="/var/lib/kubelet/pods/991d10ac-9d46-41e1-a208-d1ca31a8d488/volumes"
Oct 29 23:36:38.027482 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:38.027448    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"]
Oct 29 23:36:38.030548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:38.030528    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-97784caf-bhnnh"]
Oct 29 23:36:39.061167 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:39.061128    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f78e8e21-fb16-4eff-a071-b6b7f0b2908f" path="/var/lib/kubelet/pods/f78e8e21-fb16-4eff-a071-b6b7f0b2908f/volumes"
Oct 29 23:36:52.033423 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:52.033384    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"]
Oct 29 23:36:52.046965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:52.046934    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-1a8d697f-bflld"]
Oct 29 23:36:52.561783 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:52.561742    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:36:53.060449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:53.060416    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="145ea915-7b9c-4279-8f1f-705a1033cb0c" path="/var/lib/kubelet/pods/145ea915-7b9c-4279-8f1f-705a1033cb0c/volumes"
Oct 29 23:36:54.072394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:54.072359    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:36:54.614685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:36:54.614648    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:36:56.148739 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:36:56.148705    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:37:00.825883 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:00.825849    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:06.261635 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:06.261587    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:06.778124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:06.778072    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:06.929280 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:06.929250    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:10.393241 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.393206    5111 scope.go:122] "RemoveContainer" containerID="62a25847d484542e98cc8b123598d0a0c6e807300965a32f747bc9a719201cf7"
Oct 29 23:37:10.404886 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.404866    5111 scope.go:122] "RemoveContainer" containerID="a389b8f479c3effc6f8d4b4b11f8181f22d3c5f7072483210285eb73d4616bb6"
Oct 29 23:37:10.412929 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.412909    5111 scope.go:122] "RemoveContainer" containerID="8dc97134f99b019dc15057f25fed4075ff436889e35553dd46c5f96a605cd12e"
Oct 29 23:37:10.421108 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.421088    5111 scope.go:122] "RemoveContainer" containerID="fbc7ebe3dd8471bbbbb9e3ce4251b1deed0f914daaa6ad06978c4b8fa0925dc2"
Oct 29 23:37:10.763124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.763050    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:10.960498 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:10.960467    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:12.390927 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:12.390886    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:12.546305 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:12.546277    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:12.704990 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:12.704902    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:14.310834 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:14.310802    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:16.000950 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:16.000916    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:17.934072 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:17.934034    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:19.077801 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:19.077762    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:19.630614 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:19.630578    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:21.348276 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:21.348242    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:23.296361 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:23.296325    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:25.020833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:25.020793    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:25.776823 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:25.776788    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:26.148135 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:37:26.148102    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:37:26.291217 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:26.291179    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:26.435004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:26.434913    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:26.598695 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:26.598648    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:26.920232 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:26.920201    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:28.397918 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:28.397889    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:28.771130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:28.771065    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:30.464781 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:30.464746    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:32.287605 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:32.287569    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:33.911487 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:33.911454    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:35.602191 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:35.602154    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:37.235426 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:37.235389    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:38.937056 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:38.937018    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:40.544637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:40.544608    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:42.284600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:42.284562    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:43.970195 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:43.970144    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:45.688455 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:45.688416    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:47.474563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:47.474524    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:49.113889 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:49.113845    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:50.910062 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:50.909928    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:52.531798 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:52.531762    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:54.274866 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:54.274821    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:55.951717 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:55.951683    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:56.146770 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:37:56.146735    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:37:58.569408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:58.569375    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:37:59.298815 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:37:59.298775    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:00.922661 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:00.922604    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:01.059963 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:01.059923    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:02.623134 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:02.623103    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:02.764602 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:02.764572    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:04.634603 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:04.634537    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:06.246802 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:06.246768    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:07.640947 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:07.640912    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:07.948961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:07.948879    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:09.215452 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:09.215426    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:38:09.220252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:09.220229    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:38:09.518401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:09.518321    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:10.452154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:10.452120    5111 scope.go:122] "RemoveContainer" containerID="474c922eb4de27d7b98252f47091bd98ed6e7f09a91376652a7ed50df7fc74b1"
Oct 29 23:38:10.461756 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:10.461732    5111 scope.go:122] "RemoveContainer" containerID="5738ac3bf4a904fdd7552c7dcca05e7d60753388262d9151b7204b9ca9707864"
Oct 29 23:38:10.469836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:10.469816    5111 scope.go:122] "RemoveContainer" containerID="f74ed119fd0a81c3e04af3d23eed52dbba19d607374e13f924b44eb8c797e37b"
Oct 29 23:38:10.477952 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:10.477936    5111 scope.go:122] "RemoveContainer" containerID="53ec5ff4231ac9a9f18c3e46918e945f0bb1e5edfaeb9c88af039b7f4884777c"
Oct 29 23:38:11.204685 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:11.198039    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:12.791598 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:12.791550    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:13.498187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:13.498157    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:13.630562 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:13.630528    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:14.284603 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:14.284567    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:14.800215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:14.800183    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:14.944607 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:14.944576    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:15.434915 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:15.434885    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:21.753294 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:21.753264    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:21.896439 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:21.896408    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:22.111567 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:22.111526    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:26.146789 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:38:26.146749    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:38:30.924572 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:30.924526    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:31.916798 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:31.916766    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:37.506252 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:38:37.506214    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:38:56.149422 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:38:56.149379    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:39:01.062551 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:39:01.062512    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:39:26.149155 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:39:26.149073    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:39:56.148292 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:39:56.148251    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:40:01.158411 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:01.158376    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:06.640422 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:06.640367    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:40:06.641154 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:06.641099    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:40:26.147337 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:40:26.147301    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:40:36.646869 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:36.646832    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:37.145026 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:37.144968    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:37.327323 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:37.327277    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:42.743992 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:42.743940    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:42.897531 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:42.897495    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:43.063808 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:43.063733    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:44.394466 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:44.394427    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:46.162920 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:46.162867    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:47.838645 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:47.838609    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:49.630235 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:49.630202    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:51.389794 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:51.389710    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:53.147514 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:53.147476    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:54.820698 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:54.820648    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:55.953016 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:55.952969    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:56.147822 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:40:56.147784    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:40:56.648577 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:56.648546    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:56.839066 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:56.839034    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:58.309868 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:58.309829    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:40:58.949926 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:40:58.949888    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:00.722452 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:00.722418    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:01.256701 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:01.256666    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:02.492875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:02.492834    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:04.216199 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:04.216168    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:05.873938 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:05.873902    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:07.640484 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:07.640452    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:09.357800 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:09.357760    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:11.206720 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:11.206685    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:12.883261 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:12.883221    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:14.717259 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:14.717227    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:16.449547 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:16.449512    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:18.175412 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:18.175350    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:19.866875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:19.866836    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:21.592773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:21.592731    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:23.392637 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:23.392582    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:25.077961 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:25.077927    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:26.146845 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:41:26.146811    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:41:26.787386 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:26.787342    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:28.034552 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:28.034514    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:28.607482 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:28.607446    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:28.765698 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:28.765665    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:30.419197 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:30.419159    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:32.055311 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:32.055277    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:33.658174 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:33.658140    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:33.820888 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:33.820848    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:35.455786 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:35.455751    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:37.088806 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:37.088773    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:38.802510 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:38.802457    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:39.476332 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:39.476299    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:39.612991 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:39.612945    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:40.582317 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:40.582277    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:42.057027 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:42.056991    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:42.574452 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:42.574413    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:42.711557 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:42.711524    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:43.210682 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:43.210650    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:50.519833 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:50.519794    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:50.734228 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:50.734194    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:51.068604 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:51.068566    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:55.556433 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:41:55.556390    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:41:56.150070 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:41:56.150037    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:42:01.322688 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:01.322654    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:09.706395 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:09.706337    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:11.692642 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:11.692603    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:12.063571 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:12.063495    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:17.228077 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:17.228036    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:18.864983 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:18.864922    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:26.148464 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:42:26.148427    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:42:47.541575 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:47.541527    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:50.082862 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:50.082802    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:42:56.148290 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:42:56.148247    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:42:58.433187 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:42:58.433149    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:43:01.474779 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:01.474749    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:43:06.642106 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:06.642067    5111 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-wpwhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 29 23:43:06.642839 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:06.642806    5111 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-wpwhs" podUID="d1c4d693-e49e-41d5-935c-3e12cd99e3b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 29 23:43:09.263398 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:09.263365    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:43:09.273775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:09.273750    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:43:26.147035 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:43:26.146996    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:43:38.345772 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:38.345733    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:43:38.480542 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:38.480510    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:43:47.469988 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.469937    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"]
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470335    5111 cpu_manager.go:414] "containerMap: removing container" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy"
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470348    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy"
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470359    5111 cpu_manager.go:414] "containerMap: removing container" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy"
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470364    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy"
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470373    5111 cpu_manager.go:414] "containerMap: removing container" podUID="145ea915-7b9c-4279-8f1f-705a1033cb0c" containerName="revoke-keys"
Oct 29 23:43:47.470378 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470379    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="145ea915-7b9c-4279-8f1f-705a1033cb0c" containerName="revoke-keys"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470399    5111 cpu_manager.go:414] "containerMap: removing container" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470404    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470424    5111 cpu_manager.go:414] "containerMap: removing container" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470428    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470506    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3300e93-7cf1-407e-b46b-b43637553b86" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470514    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0cd2e02-af0d-4ed4-922f-da7fac0a3726" containerName="main"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470522    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="00c6ed7b-e070-4cd0-bfd3-949afca12fa4" containerName="istio-proxy"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470529    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="36d1780e-f31a-4f2f-a0f1-ccd0d168f50c" containerName="istio-proxy"
Oct 29 23:43:47.470612 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.470535    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="145ea915-7b9c-4279-8f1f-705a1033cb0c" containerName="revoke-keys"
Oct 29 23:43:47.473624 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.473607    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.504457 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.504431    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:43:47.504601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.504501    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:43:47.504601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.504508    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:43:47.504601 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.504528    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:43:47.506324 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.506303    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"]
Oct 29 23:43:47.642151 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.642130    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.642279 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.642156    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pld6f\" (UniqueName: \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.642279 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.642266    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.743257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.743192    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.743257 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.743250    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.743449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.743267    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pld6f\" (UniqueName: \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.743850 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.743828    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.745766 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.745742    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.763351 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.763333    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pld6f\" (UniqueName: \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\") pod \"maas-api-revoke-sub-f03629db-ts5cn\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:47.786636 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:47.786618    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:48.083299 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:48.083276    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"]
Oct 29 23:43:48.320432 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:48.320405    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn" event={"ID":"2b66a712-36b2-4c7f-9e60-bb1596c4e138","Type":"ContainerStarted","Data":"83f307e394dc634e3f50a5ce674842f4ef67c0a5ab48d0a8a95c6738e4818e6f"}
Oct 29 23:43:48.320526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:48.320462    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn" event={"ID":"2b66a712-36b2-4c7f-9e60-bb1596c4e138","Type":"ContainerStarted","Data":"3f529f52802901745527a45d04f2f4ece50963442b02d51fb1833cbee1237da4"}
Oct 29 23:43:48.363224 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:48.363154    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn" podStartSLOduration=1.363140957 podStartE2EDuration="1.363140957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:43:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:43:48.362044846 +0000 UTC m=+1839.448327980" watchObservedRunningTime="2026-10-29 23:43:48.363140957 +0000 UTC m=+1839.449424085"
Oct 29 23:43:49.326546 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:49.326514    5111 generic.go:356] "Generic (PLEG): container finished" podID="2b66a712-36b2-4c7f-9e60-bb1596c4e138" containerID="83f307e394dc634e3f50a5ce674842f4ef67c0a5ab48d0a8a95c6738e4818e6f" exitCode=0
Oct 29 23:43:49.326900 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:49.326572    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn" event={"ID":"2b66a712-36b2-4c7f-9e60-bb1596c4e138","Type":"ContainerDied","Data":"83f307e394dc634e3f50a5ce674842f4ef67c0a5ab48d0a8a95c6738e4818e6f"}
Oct 29 23:43:50.466956 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.466930    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:50.470087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.470064    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\") pod \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") "
Oct 29 23:43:50.470225 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.470107    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\" (UniqueName: \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\") pod \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") "
Oct 29 23:43:50.470225 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.470141    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\") pod \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\" (UID: \"2b66a712-36b2-4c7f-9e60-bb1596c4e138\") "
Oct 29 23:43:50.470555 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.470534    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca" pod "2b66a712-36b2-4c7f-9e60-bb1596c4e138" (UID: "2b66a712-36b2-4c7f-9e60-bb1596c4e138"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:43:50.472058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.472033    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f" pod "2b66a712-36b2-4c7f-9e60-bb1596c4e138" (UID: "2b66a712-36b2-4c7f-9e60-bb1596c4e138"). InnerVolumeSpecName "kube-api-access-pld6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:43:50.472058 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.472038    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca" pod "2b66a712-36b2-4c7f-9e60-bb1596c4e138" (UID: "2b66a712-36b2-4c7f-9e60-bb1596c4e138"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:43:50.571127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.571096    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:43:50.571127 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.571123    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/2b66a712-36b2-4c7f-9e60-bb1596c4e138-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:43:50.571296 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:50.571136    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pld6f\" (UniqueName: \"kubernetes.io/projected/2b66a712-36b2-4c7f-9e60-bb1596c4e138-kube-api-access-pld6f\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:43:51.338509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:51.338481    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn"
Oct 29 23:43:51.338509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:51.338499    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-ts5cn" event={"ID":"2b66a712-36b2-4c7f-9e60-bb1596c4e138","Type":"ContainerDied","Data":"3f529f52802901745527a45d04f2f4ece50963442b02d51fb1833cbee1237da4"}
Oct 29 23:43:51.338752 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:43:51.338548    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f529f52802901745527a45d04f2f4ece50963442b02d51fb1833cbee1237da4"
Oct 29 23:43:56.147418 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:43:56.147383    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:44:01.720768 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:01.720696    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:44:04.203988 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.203938    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"]
Oct 29 23:44:04.204587 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.204567    5111 cpu_manager.go:414] "containerMap: removing container" podUID="2b66a712-36b2-4c7f-9e60-bb1596c4e138" containerName="revoke-keys"
Oct 29 23:44:04.204630 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.204592    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b66a712-36b2-4c7f-9e60-bb1596c4e138" containerName="revoke-keys"
Oct 29 23:44:04.204727 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.204714    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b66a712-36b2-4c7f-9e60-bb1596c4e138" containerName="revoke-keys"
Oct 29 23:44:04.208063 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.208044    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.225368 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.225345    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"]
Oct 29 23:44:04.236740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.236713    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:44:04.236740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.236714    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:44:04.236917 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.236713    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:44:04.238083 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.238063    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:44:04.279116 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.279085    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.279244 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.279124    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.279310 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.279265    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8vh4\" (UniqueName: \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.379933 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.379905    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.380076 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.379941    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.380076 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.380040    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8vh4\" (UniqueName: \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.380699 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.380678    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.382418 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.382396    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.400366 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.400342    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8vh4\" (UniqueName: \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\") pod \"maas-api-revoke-sub-0165d7d2-7vmdw\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.520600 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.520537    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:04.818179 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:04.818153    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"]
Oct 29 23:44:04.819840 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:44:04.819817    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5784c5a1_491a_433c_94e7_69bbeef8d8d3.slice/crio-0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784 WatchSource:0}: Error finding container 0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784: Status 404 returned error can't find the container with id 0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784
Oct 29 23:44:05.418965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:05.418881    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw" event={"ID":"5784c5a1-491a-433c-94e7-69bbeef8d8d3","Type":"ContainerStarted","Data":"8fff54673e51363c4247e96d9e1e5713f47879e59fcc3cfa48945bd2cbb6574e"}
Oct 29 23:44:05.418965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:05.418934    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw" event={"ID":"5784c5a1-491a-433c-94e7-69bbeef8d8d3","Type":"ContainerStarted","Data":"0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784"}
Oct 29 23:44:05.459564 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:05.459520    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw" podStartSLOduration=1.459506876 podStartE2EDuration="1.459506876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:44:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:44:05.457715371 +0000 UTC m=+1856.543998496" watchObservedRunningTime="2026-10-29 23:44:05.459506876 +0000 UTC m=+1856.545790010"
Oct 29 23:44:06.427762 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:06.427725    5111 generic.go:356] "Generic (PLEG): container finished" podID="5784c5a1-491a-433c-94e7-69bbeef8d8d3" containerID="8fff54673e51363c4247e96d9e1e5713f47879e59fcc3cfa48945bd2cbb6574e" exitCode=0
Oct 29 23:44:06.428181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:06.427782    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw" event={"ID":"5784c5a1-491a-433c-94e7-69bbeef8d8d3","Type":"ContainerDied","Data":"8fff54673e51363c4247e96d9e1e5713f47879e59fcc3cfa48945bd2cbb6574e"}
Oct 29 23:44:07.566826 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.566800    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:07.608172 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.608142    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\" (UniqueName: \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\") pod \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") "
Oct 29 23:44:07.608306 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.608197    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\") pod \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") "
Oct 29 23:44:07.608306 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.608233    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\") pod \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\" (UID: \"5784c5a1-491a-433c-94e7-69bbeef8d8d3\") "
Oct 29 23:44:07.608895 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.608861    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca" pod "5784c5a1-491a-433c-94e7-69bbeef8d8d3" (UID: "5784c5a1-491a-433c-94e7-69bbeef8d8d3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:44:07.610452 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.610423    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4" pod "5784c5a1-491a-433c-94e7-69bbeef8d8d3" (UID: "5784c5a1-491a-433c-94e7-69bbeef8d8d3"). InnerVolumeSpecName "kube-api-access-j8vh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:44:07.610651 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.610630    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca" pod "5784c5a1-491a-433c-94e7-69bbeef8d8d3" (UID: "5784c5a1-491a-433c-94e7-69bbeef8d8d3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:44:07.709124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.709073    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8vh4\" (UniqueName: \"kubernetes.io/projected/5784c5a1-491a-433c-94e7-69bbeef8d8d3-kube-api-access-j8vh4\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:07.709124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.709094    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:07.709124 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:07.709104    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/5784c5a1-491a-433c-94e7-69bbeef8d8d3-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:08.438597 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:08.438560    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw" event={"ID":"5784c5a1-491a-433c-94e7-69bbeef8d8d3","Type":"ContainerDied","Data":"0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784"}
Oct 29 23:44:08.438597 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:08.438587    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-7vmdw"
Oct 29 23:44:08.438807 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:08.438617    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0183d512bbdf013329dc24c864aacdb942ea26559bebe997edcb73a384739784"
Oct 29 23:44:15.175120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.175091    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"]
Oct 29 23:44:15.175529 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.175464    5111 cpu_manager.go:414] "containerMap: removing container" podUID="5784c5a1-491a-433c-94e7-69bbeef8d8d3" containerName="revoke-keys"
Oct 29 23:44:15.175529 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.175475    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5784c5a1-491a-433c-94e7-69bbeef8d8d3" containerName="revoke-keys"
Oct 29 23:44:15.175617 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.175556    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="5784c5a1-491a-433c-94e7-69bbeef8d8d3" containerName="revoke-keys"
Oct 29 23:44:15.178684 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.178665    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.199509 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.199483    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"]
Oct 29 23:44:15.210954 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.210929    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:44:15.211122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.210989    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 29 23:44:15.211122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.211013    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:44:15.211122 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.211095    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:44:15.265921 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.265892    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75swx\" (UniqueName: \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.266087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.265943    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.266087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.266029    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.367031 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.367000    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.367208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.367138    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75swx\" (UniqueName: \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.367208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.367187    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.367669 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.367645    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.369417 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.369397    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.387577 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.387552    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75swx\" (UniqueName: \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\") pod \"maas-api-revoke-sub-6b19338f-2c2bc\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.490703 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.490652    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:15.784571 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:15.784539    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"]
Oct 29 23:44:15.787149 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:44:15.787115    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b8eb96_f163_4ea5_879d_bbba6bdd05ed.slice/crio-a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4 WatchSource:0}: Error finding container a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4: Status 404 returned error can't find the container with id a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4
Oct 29 23:44:16.477684 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:16.477606    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc" event={"ID":"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed","Type":"ContainerStarted","Data":"5ab626a6abc50c7a5d2f07598a5e3b69ab8d65b364acae155e83b4b63df09b25"}
Oct 29 23:44:16.478080 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:16.477684    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc" event={"ID":"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed","Type":"ContainerStarted","Data":"a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4"}
Oct 29 23:44:16.518271 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:16.518231    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc" podStartSLOduration=1.518218415 podStartE2EDuration="1.518218415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:44:16.517942448 +0000 UTC m=+1867.604225582" watchObservedRunningTime="2026-10-29 23:44:16.518218415 +0000 UTC m=+1867.604501549"
Oct 29 23:44:16.884986 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:16.884926    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:44:17.485042 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:17.485008    5111 generic.go:356] "Generic (PLEG): container finished" podID="b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" containerID="5ab626a6abc50c7a5d2f07598a5e3b69ab8d65b364acae155e83b4b63df09b25" exitCode=0
Oct 29 23:44:17.485446 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:17.485050    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc" event={"ID":"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed","Type":"ContainerDied","Data":"5ab626a6abc50c7a5d2f07598a5e3b69ab8d65b364acae155e83b4b63df09b25"}
Oct 29 23:44:18.634214 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.634187    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:18.693882 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.693859    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\" (UniqueName: \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\") pod \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") "
Oct 29 23:44:18.694040 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.693899    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\") pod \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") "
Oct 29 23:44:18.694040 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.694013    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\") pod \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\" (UID: \"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed\") "
Oct 29 23:44:18.694270 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.694247    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca" pod "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" (UID: "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:44:18.695958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.695931    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx" pod "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" (UID: "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed"). InnerVolumeSpecName "kube-api-access-75swx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:44:18.695958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.695938    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca" pod "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" (UID: "b7b8eb96-f163-4ea5-879d-bbba6bdd05ed"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 29 23:44:18.795419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.795362    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75swx\" (UniqueName: \"kubernetes.io/projected/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-kube-api-access-75swx\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:18.795419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.795391    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-openshift-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:18.795419 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:18.795400    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b7b8eb96-f163-4ea5-879d-bbba6bdd05ed-maas-api-xks-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:44:19.497401 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:19.497373    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc"
Oct 29 23:44:19.497536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:19.497406    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2c2bc" event={"ID":"b7b8eb96-f163-4ea5-879d-bbba6bdd05ed","Type":"ContainerDied","Data":"a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4"}
Oct 29 23:44:19.497536 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:19.497465    5111 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a626ab2c38fb72259bbcc28516698692431f4219d5e7ae333be9fb7afeb7eee4"
Oct 29 23:44:26.147931 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:44:26.147896    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:44:37.593050 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:37.593013    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-qssv2_d4c958e6-6b39-48e0-be3c-2362004a307b/manager/0.log"
Oct 29 23:44:37.792934 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:37.792894    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-qzzlh_c8440b63-eab9-4f18-99b5-6746c354b613/manager/0.log"
Oct 29 23:44:38.016281 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:38.016209    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-96hcn_f31401ea-48ef-4ffa-a946-eb312302cbbd/manager/0.log"
Oct 29 23:44:38.667526 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:38.667499    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-qwkxl_28a0d6f6-5f87-422d-b41e-ed69859aa225/server/0.log"
Oct 29 23:44:38.857407 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:38.857386    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-n5rdp_9c40a8fd-7e19-48e9-8f5d-8f43c798d235/manager/0.log"
Oct 29 23:44:39.076733 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:39.076696    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-x4vpd_02a5137b-4531-4484-9661-6257be71c966/manager/0.log"
Oct 29 23:44:39.780674 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:39.780645    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb7fb9d97-zd2fw_d47b2c1f-2018-4906-bd8c-b44ee9b8240b/manager/0.log"
Oct 29 23:44:39.828034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:39.828011    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb7fb9d97-zd2fw_d47b2c1f-2018-4906-bd8c-b44ee9b8240b/copy-manifests/0.log"
Oct 29 23:44:41.250775 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:41.250737    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-7vmdw_5784c5a1-491a-433c-94e7-69bbeef8d8d3/revoke-keys/0.log"
Oct 29 23:44:41.805790 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:41.805758    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2c2bc_b7b8eb96-f163-4ea5-879d-bbba6bdd05ed/revoke-keys/0.log"
Oct 29 23:44:42.774548 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:42.774516    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-ts5cn_2b66a712-36b2-4c7f-9e60-bb1596c4e138/revoke-keys/0.log"
Oct 29 23:44:47.290450 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:47.290414    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbcb88b9-g7xvv_3b3ff99f-81d0-4bc1-b47a-ebb26e736c02/kube-auth-proxy/0.log"
Oct 29 23:44:47.469507 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:47.469464    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-59689_1a5477d0-d8dc-421f-80f6-40029d523361/istio-proxy/0.log"
Oct 29 23:44:48.282524 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:48.282486    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-gdsfs_ed0eab29-57b8-4012-8f45-04a0914fcd73/istio-proxy/0.log"
Oct 29 23:44:48.486374 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:48.486338    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-wpwhs_d1c4d693-e49e-41d5-935c-3e12cd99e3b5/istio-proxy/0.log"
Oct 29 23:44:49.890391 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:44:49.890353    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cddcfffcc-8wmcx_d7081746-11b4-4ea2-b641-2fc60782a2cf/router/0.log"
Oct 29 23:44:56.149342 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:44:56.149300    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:45:00.143697 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.143658    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:00.144215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.144065    5111 cpu_manager.go:414] "containerMap: removing container" podUID="b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" containerName="revoke-keys"
Oct 29 23:45:00.144215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.144079    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" containerName="revoke-keys"
Oct 29 23:45:00.144215 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.144173    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7b8eb96-f163-4ea5-879d-bbba6bdd05ed" containerName="revoke-keys"
Oct 29 23:45:00.147619 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.147594    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.166449 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.164453    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 29 23:45:00.167184 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.167150    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:00.177958 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.177937    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 29 23:45:00.178113 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.178062    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-txsxx\""
Oct 29 23:45:00.264142 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.264117    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.264284 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.264250    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bllf\" (UniqueName: \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.365029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.365004    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bllf\" (UniqueName: \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.365143 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.365046    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.365626 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.365608    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.385714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.385691    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bllf\" (UniqueName: \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\") pod \"maas-api-key-cleanup-29888625-j2rd7\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.461262 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.461212    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:00.757664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:00.757641    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:00.759375 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:45:00.759345    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dab3040_0386_41fa_b389_a26e0813309f.slice/crio-de6ea359d862a543bfefcf9f2b61ac8dbdb6f4b8eba23e6e7706797f4242f630 WatchSource:0}: Error finding container de6ea359d862a543bfefcf9f2b61ac8dbdb6f4b8eba23e6e7706797f4242f630: Status 404 returned error can't find the container with id de6ea359d862a543bfefcf9f2b61ac8dbdb6f4b8eba23e6e7706797f4242f630
Oct 29 23:45:01.703160 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:01.703131    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:45:01.717309 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:01.717285    5111 generic.go:356] "Generic (PLEG): container finished" podID="4dab3040-0386-41fa-b389-a26e0813309f" containerID="015241de7a1c5789a0c88d68410eb016a7d60e9366915af08ba0f3c7d2dac59c" exitCode=60
Oct 29 23:45:01.717427 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:01.717362    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" event={"ID":"4dab3040-0386-41fa-b389-a26e0813309f","Type":"ContainerDied","Data":"015241de7a1c5789a0c88d68410eb016a7d60e9366915af08ba0f3c7d2dac59c"}
Oct 29 23:45:01.717427 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:01.717409    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" event={"ID":"4dab3040-0386-41fa-b389-a26e0813309f","Type":"ContainerStarted","Data":"de6ea359d862a543bfefcf9f2b61ac8dbdb6f4b8eba23e6e7706797f4242f630"}
Oct 29 23:45:01.717563 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:01.717550    5111 scope.go:122] "RemoveContainer" containerID="015241de7a1c5789a0c88d68410eb016a7d60e9366915af08ba0f3c7d2dac59c"
Oct 29 23:45:02.472135 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.472108    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-qssv2_d4c958e6-6b39-48e0-be3c-2362004a307b/manager/0.log"
Oct 29 23:45:02.683352 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.683289    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-5cfb77bf8c-qzzlh_c8440b63-eab9-4f18-99b5-6746c354b613/manager/0.log"
Oct 29 23:45:02.723256 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.723229    5111 generic.go:356] "Generic (PLEG): container finished" podID="4dab3040-0386-41fa-b389-a26e0813309f" containerID="7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba" exitCode=60
Oct 29 23:45:02.723660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.723293    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" event={"ID":"4dab3040-0386-41fa-b389-a26e0813309f","Type":"ContainerDied","Data":"7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba"}
Oct 29 23:45:02.723660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.723336    5111 scope.go:122] "RemoveContainer" containerID="015241de7a1c5789a0c88d68410eb016a7d60e9366915af08ba0f3c7d2dac59c"
Oct 29 23:45:02.723660 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.723557    5111 scope.go:122] "RemoveContainer" containerID="7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba"
Oct 29 23:45:02.723814 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:45:02.723778    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888625-j2rd7_odh-ai-gateway-infra(4dab3040-0386-41fa-b389-a26e0813309f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" podUID="4dab3040-0386-41fa-b389-a26e0813309f"
Oct 29 23:45:02.897339 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:02.897314    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-96hcn_f31401ea-48ef-4ffa-a946-eb312302cbbd/manager/0.log"
Oct 29 23:45:03.370848 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:03.370819    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-qwkxl_28a0d6f6-5f87-422d-b41e-ed69859aa225/server/0.log"
Oct 29 23:45:03.563330 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:03.563302    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-n5rdp_9c40a8fd-7e19-48e9-8f5d-8f43c798d235/manager/0.log"
Oct 29 23:45:03.730243 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:03.730182    5111 scope.go:122] "RemoveContainer" containerID="7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba"
Oct 29 23:45:03.730581 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:45:03.730346    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29888625-j2rd7_odh-ai-gateway-infra(4dab3040-0386-41fa-b389-a26e0813309f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" podUID="4dab3040-0386-41fa-b389-a26e0813309f"
Oct 29 23:45:03.783777 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:03.783752    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-x4vpd_02a5137b-4531-4484-9661-6257be71c966/manager/0.log"
Oct 29 23:45:04.514655 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:04.514627    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb7fb9d97-zd2fw_d47b2c1f-2018-4906-bd8c-b44ee9b8240b/copy-manifests/0.log"
Oct 29 23:45:04.547875 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:04.547840    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bb7fb9d97-zd2fw_d47b2c1f-2018-4906-bd8c-b44ee9b8240b/manager/0.log"
Oct 29 23:45:05.679161 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:05.679125    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29888625-j2rd7_4dab3040-0386-41fa-b389-a26e0813309f/cleanup/1.log"
Oct 29 23:45:05.872530 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:05.872497    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-7vmdw_5784c5a1-491a-433c-94e7-69bbeef8d8d3/revoke-keys/0.log"
Oct 29 23:45:06.430500 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:06.430467    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2c2bc_b7b8eb96-f163-4ea5-879d-bbba6bdd05ed/revoke-keys/0.log"
Oct 29 23:45:07.353384 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:07.353355    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-ts5cn_2b66a712-36b2-4c7f-9e60-bb1596c4e138/revoke-keys/0.log"
Oct 29 23:45:11.189714 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:11.189672    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbcb88b9-g7xvv_3b3ff99f-81d0-4bc1-b47a-ebb26e736c02/kube-auth-proxy/0.log"
Oct 29 23:45:11.373034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:11.373004    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-59689_1a5477d0-d8dc-421f-80f6-40029d523361/istio-proxy/0.log"
Oct 29 23:45:12.164394 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:12.164363    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-gdsfs_ed0eab29-57b8-4012-8f45-04a0914fcd73/istio-proxy/0.log"
Oct 29 23:45:12.374130 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:12.374096    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-wpwhs_d1c4d693-e49e-41d5-935c-3e12cd99e3b5/istio-proxy/0.log"
Oct 29 23:45:13.794748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:13.794716    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cddcfffcc-8wmcx_d7081746-11b4-4ea2-b641-2fc60782a2cf/router/0.log"
Oct 29 23:45:17.054748 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:17.054718    5111 scope.go:122] "RemoveContainer" containerID="7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba"
Oct 29 23:45:17.800344 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:17.800315    5111 generic.go:356] "Generic (PLEG): container finished" podID="4dab3040-0386-41fa-b389-a26e0813309f" containerID="1edea342b4e4852e8f8df9e97f684993e06a617d299b7de5ebd95814de876990" exitCode=60
Oct 29 23:45:17.800454 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:17.800389    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" event={"ID":"4dab3040-0386-41fa-b389-a26e0813309f","Type":"ContainerDied","Data":"1edea342b4e4852e8f8df9e97f684993e06a617d299b7de5ebd95814de876990"}
Oct 29 23:45:17.800496 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:17.800454    5111 scope.go:122] "RemoveContainer" containerID="7f0010503952cd05aa48530348a357d47fb718423ef346709968c4cdff6104ba"
Oct 29 23:45:17.800803 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:17.800783    5111 scope.go:122] "RemoveContainer" containerID="1edea342b4e4852e8f8df9e97f684993e06a617d299b7de5ebd95814de876990"
Oct 29 23:45:17.801021 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:45:17.800997    5111 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29888625-j2rd7_odh-ai-gateway-infra(4dab3040-0386-41fa-b389-a26e0813309f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" podUID="4dab3040-0386-41fa-b389-a26e0813309f"
Oct 29 23:45:18.853726 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:18.853688    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:19.002204 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.002176    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:19.117664 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.117593    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\") pod \"4dab3040-0386-41fa-b389-a26e0813309f\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") "
Oct 29 23:45:19.117844 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.117714    5111 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\" (UniqueName: \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\") pod \"4dab3040-0386-41fa-b389-a26e0813309f\" (UID: \"4dab3040-0386-41fa-b389-a26e0813309f\") "
Oct 29 23:45:19.118087 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.118060    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca" pod "4dab3040-0386-41fa-b389-a26e0813309f" (UID: "4dab3040-0386-41fa-b389-a26e0813309f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 29 23:45:19.119738 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.119714    5111 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf" pod "4dab3040-0386-41fa-b389-a26e0813309f" (UID: "4dab3040-0386-41fa-b389-a26e0813309f"). InnerVolumeSpecName "kube-api-access-4bllf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 29 23:45:19.218596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.218561    5111 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bllf\" (UniqueName: \"kubernetes.io/projected/4dab3040-0386-41fa-b389-a26e0813309f-kube-api-access-4bllf\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:45:19.218596 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.218593    5111 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4dab3040-0386-41fa-b389-a26e0813309f-maas-api-service-ca\") on node \"ip-10-0-141-12.ec2.internal\" DevicePath \"\""
Oct 29 23:45:19.822710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.822671    5111 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"
Oct 29 23:45:19.822710 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.822692    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7" event={"ID":"4dab3040-0386-41fa-b389-a26e0813309f","Type":"ContainerDied","Data":"de6ea359d862a543bfefcf9f2b61ac8dbdb6f4b8eba23e6e7706797f4242f630"}
Oct 29 23:45:19.823008 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.822774    5111 scope.go:122] "RemoveContainer" containerID="1edea342b4e4852e8f8df9e97f684993e06a617d299b7de5ebd95814de876990"
Oct 29 23:45:19.878208 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.878176    5111 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:19.891788 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:19.891765    5111 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29888625-j2rd7"]
Oct 29 23:45:21.060773 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:45:21.060675    5111 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4dab3040-0386-41fa-b389-a26e0813309f" path="/var/lib/kubelet/pods/4dab3040-0386-41fa-b389-a26e0813309f/volumes"
Oct 29 23:45:26.146750 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:45:26.146719    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:45:56.148364 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:45:56.148327    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:46:01.822320 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:01.822284    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:46:13.313610 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:13.313556    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:46:26.148188 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:46:26.148148    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:46:36.215488 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:36.215453    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:46:42.591176 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:42.591140    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:46:42.743038 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:42.743004    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:46:47.123341 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:47.123305    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-64m88_e29dccc7-5622-4583-b74e-220df6ff5a34/global-pull-secret-syncer/0.log"
Oct 29 23:46:47.452841 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:47.452750    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-88q8b_86fa6768-086e-4434-9d30-cc860a28acc6/connection-checker/0.log"
Oct 29 23:46:47.560174 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:47.560143    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-c68qr_615ee2a5-4bf0-40da-a0a2-3ddc0cd6d6e7/connection-checker/0.log"
Oct 29 23:46:47.671037 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:47.671009    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-hhgrs_7bd53292-a0c6-41a8-8a39-41a27853cb73/connection-checker/0.log"
Oct 29 23:46:47.791849 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:47.791757    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log"
Oct 29 23:46:48.420553 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:48.420521    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-12.ec2.internal_9bc940380ed911d9e6c421c48f4b1a6a/haproxy/0.log"
Oct 29 23:46:56.148789 ip-10-0-141-12 kubenswrapper[5111]: E1029 23:46:56.148697    5111 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mcxnv" containerName="extractor"
Oct 29 23:46:56.955226 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955190    5111 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"]
Oct 29 23:46:56.955644 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955626    5111 cpu_manager.go:414] "containerMap: removing container" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.955740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955645    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.955740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955686    5111 cpu_manager.go:414] "containerMap: removing container" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.955740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955694    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.955740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955713    5111 cpu_manager.go:414] "containerMap: removing container" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.955740 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955720    5111 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.956017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955818    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.956017 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.955831    5111 memory_manager.go:365] "RemoveStaleState removing state" podUID="4dab3040-0386-41fa-b389-a26e0813309f" containerName="cleanup"
Oct 29 23:46:56.959181 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.959161    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:56.976780 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:56.976749    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"]
Oct 29 23:46:57.044496 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.044472    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbnpg\"/\"kube-root-ca.crt\""
Oct 29 23:46:57.044652 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.044629    5111 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbnpg\"/\"openshift-service-ca.crt\""
Oct 29 23:46:57.063403 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.063385    5111 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gbnpg\"/\"default-dockercfg-kv5q7\""
Oct 29 23:46:57.103916 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.103892    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-sys\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.104029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.103922    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-podres\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.104029 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.103946    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-proc\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.104120 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.104027    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-lib-modules\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.104157 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.104121    5111 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k46q\" (UniqueName: \"kubernetes.io/projected/1467f6e9-c10c-4a89-9939-1addbf2baa64-kube-api-access-4k46q\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205004 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.204960    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k46q\" (UniqueName: \"kubernetes.io/projected/1467f6e9-c10c-4a89-9939-1addbf2baa64-kube-api-access-4k46q\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205041    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-sys\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205095    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-sys\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205343    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-podres\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205375    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-proc\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205401    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-podres\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205441 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205410    5111 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-lib-modules\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205449    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-proc\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.205668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.205475    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1467f6e9-c10c-4a89-9939-1addbf2baa64-lib-modules\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.225832 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.225803    5111 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k46q\" (UniqueName: \"kubernetes.io/projected/1467f6e9-c10c-4a89-9939-1addbf2baa64-kube-api-access-4k46q\") pod \"perf-node-gather-daemonset-22r8r\" (UID: \"1467f6e9-c10c-4a89-9939-1addbf2baa64\") " pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.271303 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.271288    5111 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:57.565937 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:57.565913    5111 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"]
Oct 29 23:46:57.567462 ip-10-0-141-12 kubenswrapper[5111]: W1029 23:46:57.567424    5111 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1467f6e9_c10c_4a89_9939_1addbf2baa64.slice/crio-0c7c684dd763e055aab9c037760e468a30244c2bd7782a5fd4c7eaf01f1eeeb7 WatchSource:0}: Error finding container 0c7c684dd763e055aab9c037760e468a30244c2bd7782a5fd4c7eaf01f1eeeb7: Status 404 returned error can't find the container with id 0c7c684dd763e055aab9c037760e468a30244c2bd7782a5fd4c7eaf01f1eeeb7
Oct 29 23:46:58.307836 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:58.307805    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r" event={"ID":"1467f6e9-c10c-4a89-9939-1addbf2baa64","Type":"ContainerStarted","Data":"42698110fae99a1d35ff0da899ecb53d060ab7a2bc2d49663e7a4c6436066049"}
Oct 29 23:46:58.308255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:58.307859    5111 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r" event={"ID":"1467f6e9-c10c-4a89-9939-1addbf2baa64","Type":"ContainerStarted","Data":"0c7c684dd763e055aab9c037760e468a30244c2bd7782a5fd4c7eaf01f1eeeb7"}
Oct 29 23:46:58.308255 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:58.307886    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:46:58.350111 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:58.350058    5111 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r" podStartSLOduration=2.350042232 podStartE2EDuration="2.350042232s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:46:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:46:58.349748236 +0000 UTC m=+2029.436031377" watchObservedRunningTime="2026-10-29 23:46:58.350042232 +0000 UTC m=+2029.436325367"
Oct 29 23:46:58.358965 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:58.358944    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-9wdnw_aae915ed-0747-410a-acec-a18068a98efe/cluster-monitoring-operator/0.log"
Oct 29 23:46:59.407408 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:59.407379    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b8zsd_9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d/node-exporter/0.log"
Oct 29 23:46:59.513990 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:59.513959    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b8zsd_9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d/kube-rbac-proxy/0.log"
Oct 29 23:46:59.622593 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:46:59.622569    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b8zsd_9d84fbaf-f8de-4a9d-9b1d-f0fe11601c6d/init-textfile/0.log"
Oct 29 23:47:01.520568 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:01.520535    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-rpvlz_1eef2e78-c3b9-4f5f-bc38-33e67ef18f35/prometheus-operator-admission-webhook/0.log"
Oct 29 23:47:01.903360 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:01.903330    5111 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9h9g_7e1cae4c-99a8-4eb9-a957-1b38f9460e55/konnectivity-agent/0.log" limit=1024
Oct 29 23:47:03.887517 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:03.887491    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-n72kb_47425113-93db-453d-ac5e-b1b778b9bfe3/networking-console-plugin/0.log"
Oct 29 23:47:04.323468 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:04.323442    5111 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gbnpg/perf-node-gather-daemonset-22r8r"
Oct 29 23:47:04.891073 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:04.891040    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/1.log"
Oct 29 23:47:04.896075 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:04.896045    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-7v65t_c765f10e-91f1-4c9f-9ffd-2740b2bde8b9/console-operator/2.log"
Oct 29 23:47:05.999736 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:05.999704    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-zhng8_64e184bf-833d-4628-826e-001799a103b5/download-server/0.log"
Oct 29 23:47:07.250707 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:07.250678    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-g2czn_f5c0147f-93d0-431f-a294-93c26838eff3/volume-data-source-validator/0.log"
Oct 29 23:47:09.691360 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:09.691318    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8xtp_d42a6856-295d-4cf1-8a6e-691fcc073a34/dns/0.log"
Oct 29 23:47:09.797211 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:09.797186    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f8xtp_d42a6856-295d-4cf1-8a6e-691fcc073a34/kube-rbac-proxy/0.log"
Oct 29 23:47:10.128668 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:10.128641    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4blvf_e5f497d2-8302-4707-b479-4dca872829e3/dns-node-resolver/0.log"
Oct 29 23:47:10.928229 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:10.928196    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbcb88b9-g7xvv_3b3ff99f-81d0-4bc1-b47a-ebb26e736c02/kube-auth-proxy/0.log"
Oct 29 23:47:10.997532 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:10.997503    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-59689_1a5477d0-d8dc-421f-80f6-40029d523361/istio-proxy/0.log"
Oct 29 23:47:11.223798 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:11.223727    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-gdsfs_ed0eab29-57b8-4012-8f45-04a0914fcd73/istio-proxy/0.log"
Oct 29 23:47:11.287050 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:11.287021    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-wpwhs_d1c4d693-e49e-41d5-935c-3e12cd99e3b5/istio-proxy/0.log"
Oct 29 23:47:11.372099 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:11.372050    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74d994776-2n5gg_060116d8-5fc1-43b6-9f4d-0ffc236c4372/registry/0.log"
Oct 29 23:47:11.474994 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:11.474908    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g9m5t_d8081555-b5de-4ffa-bbe7-e2931b269310/node-ca/0.log"
Oct 29 23:47:11.702034 ip-10-0-141-12 kubenswrapper[5111]: I1029 23:47:11.702010    5111 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cddcfffcc-8wmcx_d7081746-11b4-4ea2-b641-2fc60782a2cf/router/0.log"