Jul 06 23:54:59.604692 ip-10-0-142-92 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 23:54:59.604707 ip-10-0-142-92 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 23:54:59.604717 ip-10-0-142-92 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 23:54:59.605034 ip-10-0-142-92 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 23:55:08.707547 ip-10-0-142-92 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 23:55:08.707557 ip-10-0-142-92 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 823a727b74f64cfd91e9466c6ad64f24 --
Jul 06 23:56:54.570583 ip-10-0-142-92 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 23:56:54.955510 ip-10-0-142-92 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:54.955510 ip-10-0-142-92 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 23:56:54.955510 ip-10-0-142-92 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:54.955510 ip-10-0-142-92 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 23:56:54.955510 ip-10-0-142-92 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 23:56:54.956320 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.956185    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 23:56:54.960595 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960569    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.960595 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960594    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.960595 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960599    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.960595 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960602    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960608    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960613    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960616    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960619    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960622    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960625    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960630    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960633    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960636    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960640    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960642    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960646    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960649    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960651    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960654    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960657    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960659    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960662    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.960717 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960664    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960667    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960669    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960671    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960674    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960677    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960679    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960682    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960684    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960687    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960689    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960692    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960695    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960697    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960700    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960703    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960706    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960708    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960711    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960713    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.961150 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960716    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960719    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960722    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960724    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960727    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960730    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960733    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960735    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960737    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960741    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960744    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960746    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960748    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960751    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960754    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960756    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960759    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960761    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960763    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.961667 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960766    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960768    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960771    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960773    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960776    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960778    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960781    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960784    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960787    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960789    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960792    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960795    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960797    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960800    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960803    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960806    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960809    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960811    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960814    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960816    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.962118 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960819    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960822    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960824    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960826    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960829    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960831    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960834    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960836    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960839    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960841    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960844    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960846    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960851    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960854    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960857    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960859    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960861    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.962596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.960864    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.963132 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963120    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.963132 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963131    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963136    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963139    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963142    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963145    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963148    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963151    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963153    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963156    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963159    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963161    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963164    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963167    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963169    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963172    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963175    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963178    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963180    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963182    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963185    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.963185 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963188    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963191    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963193    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963196    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963199    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963203    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963207    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963210    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963213    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963216    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963218    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963221    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963223    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963227    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963229    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963232    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963234    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963236    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963239    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.963716 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963241    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963244    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963246    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963249    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963251    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963254    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963258    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963261    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963264    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963267    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963269    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963271    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963274    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963277    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963279    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963283    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963285    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963301    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963304    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963306    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.964154 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963309    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963311    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963314    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963317    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963319    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963322    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963324    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963327    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963331    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963334    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963337    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963340    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963342    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963345    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963347    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963350    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963352    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963355    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963359    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.964653 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963362    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963365    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963368    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963370    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963373    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963376    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963378    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963380    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963383    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963387    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963389    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963392    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963395    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963397    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963400    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963402    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963415    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963419    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963422    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963424    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.965106 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963794    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963798    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963801    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963804    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963806    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963809    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963812    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963814    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963817    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963819    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963822    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963825    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963828    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963832    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963836    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963839    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963842    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963845    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963847    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963850    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.965596 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963853    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963856    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963858    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963861    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963863    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963866    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963869    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963871    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963874    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963876    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963878    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963881    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963884    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963886    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963888    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963891    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963894    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963896    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963899    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963901    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.966073 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963904    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963906    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963909    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963911    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963914    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963917    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963920    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963923    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963926    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963928    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963931    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963933    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963937    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963939    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963942    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963945    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963949    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963952    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963955    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963958    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.966552 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963960    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963962    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963965    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963967    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963970    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963972    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963976    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963981    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963983    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963986    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963989    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963991    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963994    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963996    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.963999    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964001    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964004    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964007    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964009    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.967017 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964012    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964014    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964017    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964019    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964022    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964026    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964029    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964031    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964033    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964036    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964039    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964041    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964044    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964046    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964048    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964051    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964053    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964056    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964058    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964061    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.967483 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964137    2583 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964144    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964151    2583 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964156    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964160    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964164    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964168    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964172    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964175    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964178    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964182    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964185    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964188    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964191    2583 flags.go:64] FLAG: --cgroup-root=""
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964195    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964198    2583 flags.go:64] FLAG: --client-ca-file=""
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964201    2583 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964204    2583 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964209    2583 flags.go:64] FLAG: --cluster-domain=""
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964212    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964215    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964218    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964221    2583 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964225    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 23:56:54.967944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964228    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964231    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964234    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964237    2583 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964240    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964243    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964246    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964249    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964254    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964257    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964260    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964262    2583 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964265    2583 flags.go:64] FLAG: --enable-server="true"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964268    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964273    2583 flags.go:64] FLAG: --event-burst="100"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964276    2583 flags.go:64] FLAG: --event-qps="50"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964279    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964282    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964285    2583 flags.go:64] FLAG: --eviction-hard=""
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964301    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964304    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964307    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964310    2583 flags.go:64] FLAG: --eviction-soft=""
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964314    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964317    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 23:56:54.968522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964320    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964323    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964326    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964329    2583 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964331    2583 flags.go:64] FLAG: --feature-gates=""
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964335    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964338    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964341    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964344    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964347    2583 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964350    2583 flags.go:64] FLAG: --help="false"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964352    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-142-92.ec2.internal"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964355    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964358    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964361    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964365    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964369    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964372    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964374    2583 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964377    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964380    2583 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964383    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964386    2583 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964389    2583 flags.go:64] FLAG: --kube-reserved=""
Jul 06 23:56:54.969164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964391    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964394    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964397    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964400    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964404    2583 flags.go:64] FLAG: --lock-file=""
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964406    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964409    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964413    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964418    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964421    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964424    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964426    2583 flags.go:64] FLAG: --logging-format="text"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964429    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964433    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964435    2583 flags.go:64] FLAG: --manifest-url=""
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964438    2583 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964442    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964445    2583 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964449    2583 flags.go:64] FLAG: --max-pods="110"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964452    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964455    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964458    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964461    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964464    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964467    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 23:56:54.969743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964470    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964477    2583 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964479    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964482    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964485    2583 flags.go:64] FLAG: --pod-cidr=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964488    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964493    2583 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964496    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964499    2583 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964502    2583 flags.go:64] FLAG: --port="10250"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964505    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964508    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-007d1d0ee6a116014"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964511    2583 flags.go:64] FLAG: --qos-reserved=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964513    2583 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964516    2583 flags.go:64] FLAG: --register-node="true"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964519    2583 flags.go:64] FLAG: --register-with-taints=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964523    2583 flags.go:64] FLAG: --registry-burst="10"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964526    2583 flags.go:64] FLAG: --registry-qps="5"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964528    2583 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964531    2583 flags.go:64] FLAG: --reserved-memory=""
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964535    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964537    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964540    2583 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964543    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964546    2583 flags.go:64] FLAG: --runonce="false"
Jul 06 23:56:54.970359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964548    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964552    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964555    2583 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964557    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964560    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964563    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964566    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964569    2583 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964572    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964575    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964578    2583 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964581    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964584    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964587    2583 flags.go:64] FLAG: --system-cgroups=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964589    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964597    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964600    2583 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964603    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964607    2583 flags.go:64] FLAG: --tls-min-version=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964610    2583 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964613    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964616    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964619    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964622    2583 flags.go:64] FLAG: --v="2"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964626    2583 flags.go:64] FLAG: --version="false"
Jul 06 23:56:54.970935 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964630    2583 flags.go:64] FLAG: --vmodule=""
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964634    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.964637    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964749    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964752    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964756    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964759    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964762    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964764    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964767    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964769    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964776    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964779    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964781    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964784    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964787    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964790    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964792    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964795    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964798    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.971556 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964800    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964803    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964806    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964809    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964812    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964814    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964817    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964819    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964822    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964825    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964827    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964831    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964833    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964836    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964838    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964841    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964843    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964846    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964848    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964851    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.972113 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964854    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964856    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964859    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964863    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964865    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964868    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964870    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964873    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964876    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964880    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964883    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964886    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964889    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964892    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964896    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964899    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964902    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964905    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964907    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.972942 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964910    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964912    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964915    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964917    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964920    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964923    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964926    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964928    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964931    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964933    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964936    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964939    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964942    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964944    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964946    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964949    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964952    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964955    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964957    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964960    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.973504 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964962    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964965    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964968    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964970    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964974    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964977    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964979    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964984    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964986    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964989    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964992    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964994    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964997    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.964999    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965002    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965005    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965007    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965010    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965013    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.973987 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965017    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.974455 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965020    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.974455 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965023    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.974455 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.965025    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.974455 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.965623    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:54.974561 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.974468    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 23:56:54.974561 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.974484    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 23:56:54.974561 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.974505    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 23:56:54.974561 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.974515    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974580    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974586    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974589    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974593    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974596    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974599    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974602    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974606    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974611    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974615    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974618    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974621    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974623    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974626    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974628    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974631    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974633    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974636    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974638    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.974663 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974642    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974645    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974647    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974650    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974652    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974655    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974659    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974662    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974665    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974668    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974671    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974674    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974677    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974681    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974685    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974689    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974693    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974695    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974698    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.975145 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974700    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974703    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974707    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974711    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974714    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974716    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974719    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974721    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974725    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974727    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974731    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974735    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974738    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974741    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974744    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974748    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974751    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974753    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974757    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.975611 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974761    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974766    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974770    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974773    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974776    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974778    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974780    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974783    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974786    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974788    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974791    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974793    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974796    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974798    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974801    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974804    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974806    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974809    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974811    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974814    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.976061 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974817    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974819    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974822    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974825    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974828    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974830    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974834    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974838    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974842    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974846    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974849    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974851    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974854    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974857    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974859    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974861    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974864    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974867    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974869    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974872    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.976549 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974874    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.974877    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.974881    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975007    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975013    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975016    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975019    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975022    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975025    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975027    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975030    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975033    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975036    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975038    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975041    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 23:56:54.977025 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975044    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975047    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975050    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975052    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975055    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975057    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975060    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975062    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975066    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975070    2583 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975074    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975078    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975081    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975084    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975087    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975089    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975091    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975094    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975096    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975099    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 23:56:54.977411 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975101    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975104    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975106    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975109    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975111    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975114    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975116    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975119    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975121    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975124    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975126    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975128    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975131    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975135    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975137    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975139    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975143    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975147    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975151    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 23:56:54.977886 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975156    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975158    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975161    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975165    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975169    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975172    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975175    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975178    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975180    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975183    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975185    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975187    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975190    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975193    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975196    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975198    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975201    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975205    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 23:56:54.978451 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975208    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975212    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975215    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975219    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975223    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975228    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975232    2583 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975235    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975238    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975241    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975244    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975247    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975249    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975253    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975255    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975257    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975260    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975262    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975265    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975267    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 23:56:54.978878 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975270    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975272    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975275    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975277    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975280    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975282    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975285    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975300    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975306    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:54.975309    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.975314    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.975438    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.977567    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.978699    2583 server.go:1019] "Starting client certificate rotation"
Jul 06 23:56:54.979481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.978796    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:54.979826 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.978833    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 23:56:54.980871 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.980857    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:54.982828 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.982811    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 23:56:54.991565 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.991546    2583 log.go:25] "Validated CRI v1 runtime API"
Jul 06 23:56:54.993792 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.993777    2583 log.go:25] "Validated CRI v1 image API"
Jul 06 23:56:54.994683 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.994660    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 23:56:54.999229 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.999203    2583 fs.go:135] Filesystem UUIDs: map[0b57cea0-b0dc-4866-aedf-427f6a3643fa:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9a852748-8648-4bf1-89dc-4121f3cc382e:/dev/nvme0n1p4]
Jul 06 23:56:54.999314 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:54.999227    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 23:56:55.005088 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.004976    2583 manager.go:216] Machine: {Timestamp:2026-07-06 23:56:55.004485787 +0000 UTC m=+0.329489062 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099725 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23de7e6a2ed42ed5bf0f9137303c93 SystemUUID:ec23de7e-6a2e-d42e-d5bf-0f9137303c93 BootID:823a727b-74f6-4cfd-91e9-466c6ad64f24 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ed:3a:dc:73:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ed:3a:dc:73:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:e9:2e:11:39:3d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 23:56:55.005088 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.005077    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 23:56:55.005248 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.005186    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 23:56:55.007448 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.007423    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 23:56:55.007606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.007451    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-92.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 23:56:55.008652 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.008640    2583 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 23:56:55.008714 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.008660    2583 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 23:56:55.008714 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.008671    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:55.009221 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.009208    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 23:56:55.010399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.010388    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 23:56:55.012240 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.012223    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:55.013161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013148    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:55.013325 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013314    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 23:56:55.013381 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013354    2583 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 23:56:55.013381 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013369    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 23:56:55.013474 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013384    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 23:56:55.013474 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013399    2583 kubelet.go:398] "Adding apiserver pod source"
Jul 06 23:56:55.013474 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013412    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 23:56:55.013819 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013805    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:55.013882 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.013827    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 23:56:55.016158 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.016145    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 23:56:55.018646 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.018632    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 23:56:55.018709 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.018660    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 23:56:55.019555 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019545    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 23:56:55.019592 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019562    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 23:56:55.019592 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019569    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 23:56:55.019592 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019575    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 23:56:55.019592 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019587    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019593    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019605    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019642    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019653    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019665    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019681    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 23:56:55.019696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019686    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 23:56:55.019882 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019718    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 23:56:55.019882 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.019724    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 23:56:55.022739 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.022727    2583 server.go:1268] "Started kubelet"
Jul 06 23:56:55.022899 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.022839    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 23:56:55.022959 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.022901    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 23:56:55.023561 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.023525    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 23:56:55.023603 ip-10-0-142-92 systemd[1]: Started Kubernetes Kubelet.
Jul 06 23:56:55.024177 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.023983    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 23:56:55.024177 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.023986    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 23:56:55.024177 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.024070    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-92.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 23:56:55.024478 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.024465    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 23:56:55.025774 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.025761    2583 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 23:56:55.027503 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.026726    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-92.ec2.internal.18bfd8ad012d71ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-92.ec2.internal,UID:ip-10-0-142-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-92.ec2.internal,},FirstTimestamp:2026-07-06 23:56:55.022703022 +0000 UTC m=+0.347706294,LastTimestamp:2026-07-06 23:56:55.022703022 +0000 UTC m=+0.347706294,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-92.ec2.internal,}"
Jul 06 23:56:55.029591 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.029572    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 23:56:55.029683 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.029607    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:55.029736 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.029707    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 23:56:55.029786 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.029749    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.030177 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030150    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 23:56:55.030177 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030153    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 23:56:55.030177 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030172    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 23:56:55.030335 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030275    2583 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 23:56:55.030335 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030302    2583 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 23:56:55.030441 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030423    2583 factory.go:55] Registering systemd factory
Jul 06 23:56:55.030441 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030435    2583 factory.go:223] Registration of the systemd container factory successfully
Jul 06 23:56:55.030708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030695    2583 factory.go:153] Registering CRI-O factory
Jul 06 23:56:55.030708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030710    2583 factory.go:223] Registration of the crio container factory successfully
Jul 06 23:56:55.030801 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030772    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 23:56:55.030801 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030800    2583 factory.go:103] Registering Raw factory
Jul 06 23:56:55.030889 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.030812    2583 manager.go:1196] Started watching for new ooms in manager
Jul 06 23:56:55.032164 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.031730    2583 manager.go:349] Starting recovery of all containers
Jul 06 23:56:55.032164 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.031903    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 23:56:55.032164 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.032117    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-92.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 23:56:55.043215 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.043190    2583 manager.go:354] Recovery completed
Jul 06 23:56:55.047437 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.047257    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.049188 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049168    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.049257 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049202    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.049257 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049217    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.049787 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049775    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 23:56:55.049787 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049785    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 23:56:55.049905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049799    2583 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 23:56:55.049905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.049816    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.050810 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050796    2583 policy_none.go:49] "None policy: Start"
Jul 06 23:56:55.050905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050815    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 23:56:55.050905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050828    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 23:56:55.050905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050828    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.050997 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050920    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.050997 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.050934    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.051892 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.051881    2583 policy_none.go:47] "Start"
Jul 06 23:56:55.058977 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.058917    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-92.ec2.internal.18bfd8ad02c191e0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-92.ec2.internal,UID:ip-10-0-142-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-92.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-92.ec2.internal,},FirstTimestamp:2026-07-06 23:56:55.049187808 +0000 UTC m=+0.374191080,LastTimestamp:2026-07-06 23:56:55.049187808 +0000 UTC m=+0.374191080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-92.ec2.internal,}"
Jul 06 23:56:55.063443 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.063413    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w7fnk"
Jul 06 23:56:55.069811 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.069749    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-92.ec2.internal.18bfd8ad02c1f0a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-92.ec2.internal,UID:ip-10-0-142-92.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-92.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-92.ec2.internal,},FirstTimestamp:2026-07-06 23:56:55.049212072 +0000 UTC m=+0.374215344,LastTimestamp:2026-07-06 23:56:55.049212072 +0000 UTC m=+0.374215344,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-92.ec2.internal,}"
Jul 06 23:56:55.074340 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.074324    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-w7fnk"
Jul 06 23:56:55.081816 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.081804    2583 manager.go:337] "Starting Device Plugin manager"
Jul 06 23:56:55.081891 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.081831    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 23:56:55.081891 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.081839    2583 server.go:87] "Starting device plugin registration server"
Jul 06 23:56:55.082016 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.082006    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 23:56:55.082081 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.082017    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 23:56:55.082133 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.082079    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 23:56:55.082176 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.082153    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 23:56:55.082176 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.082161    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 23:56:55.082733 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.082714    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 23:56:55.082799 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.082763    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.150604 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.150559    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 23:56:55.152466 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.151656    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 23:56:55.152466 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.151692    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 23:56:55.152466 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.151714    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 23:56:55.152466 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.151747    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 23:56:55.154601 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.154585    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:55.182792 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.182751    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.184099 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.184074    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.184195 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.184116    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.184195 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.184128    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.184195 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.184156    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.194209 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.194189    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.194273 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.194215    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-92.ec2.internal\": node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.220983 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.220919    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.252749 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.252727    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"]
Jul 06 23:56:55.252832 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.252786    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.255857 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.255844    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.255923 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.255870    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.255923 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.255880    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.257846 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.257834    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.257996 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.257984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.258072 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258009    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.258676 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258660    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.258750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258686    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.258750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258689    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.258750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258699    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.258750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258708    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.258750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.258717    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.260712 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.260697    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.260783 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.260725    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 23:56:55.261463 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.261448    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 23:56:55.261520 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.261474    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 23:56:55.261520 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.261483    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeHasSufficientPID"
Jul 06 23:56:55.284868 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.284850    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-92.ec2.internal\" not found" node="ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.288833 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.288819    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-92.ec2.internal\" not found" node="ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.321747 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.321715    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.331386 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.331364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.331469 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.331388    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.331469 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.331403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/936b88c7565d9c88f04260646b598913-config\") pod \"kube-apiserver-proxy-ip-10-0-142-92.ec2.internal\" (UID: \"936b88c7565d9c88f04260646b598913\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.422190 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.422155    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.432439 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.432529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432444    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.432529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432461    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/936b88c7565d9c88f04260646b598913-config\") pod \"kube-apiserver-proxy-ip-10-0-142-92.ec2.internal\" (UID: \"936b88c7565d9c88f04260646b598913\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.432529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432500    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/936b88c7565d9c88f04260646b598913-config\") pod \"kube-apiserver-proxy-ip-10-0-142-92.ec2.internal\" (UID: \"936b88c7565d9c88f04260646b598913\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.432529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432513    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.432529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.432521    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15ea2d76c80edf5184bf6ff447bd6336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal\" (UID: \"15ea2d76c80edf5184bf6ff447bd6336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.522787 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.522702    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.588066 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.588034    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.591187 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.591168    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:55.623687 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.623657    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.724118 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.724086    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.824469 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.824396    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.838737 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.838713    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:55.924990 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:55.924962    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:55.978418 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.978389    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 23:56:55.978910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.978541    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:56:55.978910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:55.978562    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 23:56:56.025067 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:56.025032    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-92.ec2.internal\" not found"
Jul 06 23:56:56.038507 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:56.038473    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ea2d76c80edf5184bf6ff447bd6336.slice/crio-d2edc3644f353bc44d35fbd3cc241069b1989b2017a567a570e0dabb613d37db WatchSource:0}: Error finding container d2edc3644f353bc44d35fbd3cc241069b1989b2017a567a570e0dabb613d37db: Status 404 returned error can't find the container with id d2edc3644f353bc44d35fbd3cc241069b1989b2017a567a570e0dabb613d37db
Jul 06 23:56:56.038743 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:56.038722    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936b88c7565d9c88f04260646b598913.slice/crio-5352c2812c5bea2d10faba6cfbc0ef6593966c8c93d0fdc756e167b7ece1ff60 WatchSource:0}: Error finding container 5352c2812c5bea2d10faba6cfbc0ef6593966c8c93d0fdc756e167b7ece1ff60: Status 404 returned error can't find the container with id 5352c2812c5bea2d10faba6cfbc0ef6593966c8c93d0fdc756e167b7ece1ff60
Jul 06 23:56:56.044304 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.044267    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 23:56:56.076602 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.076525    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:51:54 +0000 UTC" deadline="2028-01-06 13:09:02.094166398 +0000 UTC"
Jul 06 23:56:56.076602 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.076552    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13165h12m6.017617105s"
Jul 06 23:56:56.123321 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.123284    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:56.130251 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.130231    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:56.151651 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.151629    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:56.152396 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.152384    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal"
Jul 06 23:56:56.154529 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.154489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal" event={"ID":"15ea2d76c80edf5184bf6ff447bd6336","Type":"ContainerStarted","Data":"d2edc3644f353bc44d35fbd3cc241069b1989b2017a567a570e0dabb613d37db"}
Jul 06 23:56:56.155509 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.155489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal" event={"ID":"936b88c7565d9c88f04260646b598913","Type":"ContainerStarted","Data":"5352c2812c5bea2d10faba6cfbc0ef6593966c8c93d0fdc756e167b7ece1ff60"}
Jul 06 23:56:56.164125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.164104    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 23:56:56.578051 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:56.578025    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:57.014167 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.014137    2583 apiserver.go:52] "Watching apiserver"
Jul 06 23:56:57.021898 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.021866    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:57.023127 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.023083    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 23:56:57.023486 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.023463    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-j6rtl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv","openshift-cluster-node-tuning-operator/tuned-b9khx","openshift-dns/node-resolver-crvxn","openshift-network-diagnostics/network-check-target-fxpmb","openshift-network-operator/iptables-alerter-k4rzb","openshift-ovn-kubernetes/ovnkube-node-x56f2","kube-system/global-pull-secret-syncer-hxjb2","kube-system/konnectivity-agent-bbcqn","kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal","openshift-image-registry/node-ca-c5v2z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal","openshift-multus/multus-additional-cni-plugins-5zdcz","openshift-multus/multus-knjcv"]
Jul 06 23:56:57.026190 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.026157    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.028723 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.028678    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.028723 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.028696    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 23:56:57.028723 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.028721    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 23:56:57.028969 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.028701    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 23:56:57.030074 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030053    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 23:56:57.030221 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030053    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.030221 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030186    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lkwqk\""
Jul 06 23:56:57.030360 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030317    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.030444 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030431    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 23:56:57.030911 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.030892    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.033677 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.033654    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.035784 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.034219    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.035889    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.036255    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.036489    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sf2zg\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.036647    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.036490    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 23:56:57.037050 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.036702    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 23:56:57.037381 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.037165    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xs2x2\""
Jul 06 23:56:57.037381 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.037306    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.039219 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.039201    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:57.039325 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.039270    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:56:57.039918 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.039893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-log-socket\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040013 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.039927    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-ovn\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040013 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.039952    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-netd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040013 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.039976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4hmr\" (UniqueName: \"kubernetes.io/projected/88bbc328-c0d6-415d-957a-b3d9a9c65aba-kube-api-access-x4hmr\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.040013 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-conf\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-systemd\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040064    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-tmp\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040096    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-system-cni-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040123    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-modprobe-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040146    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-sys\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040168    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-node-log\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040190    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-kubelet\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-etc-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040333    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-env-overrides\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-script-lib\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040436    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040464    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-tuned\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040490    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-config\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040513    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovn-node-metrics-cert\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-os-release\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040583    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysconfig\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.040645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040667    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-var-lib-kubelet\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-systemd-units\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040715    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-slash\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-host\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzdxl\" (UniqueName: \"kubernetes.io/projected/e83493b9-7ec7-4e8b-99f3-22cae75164ca-kube-api-access-xzdxl\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040818    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-netns\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29hbv\" (UniqueName: \"kubernetes.io/projected/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-kube-api-access-29hbv\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040865    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040910    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040946    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-run\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040969    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-systemd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.040994    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-var-lib-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041015    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-kubernetes\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-lib-modules\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.041143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041100    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041762 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-bin\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.041762 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cnibin\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.041762 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.041723    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.043423 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.043374    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.043847 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.043823    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.046350 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.046334    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.046438 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.046412    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:56:57.048107 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048084    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.048107 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048100    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ztzj5\""
Jul 06 23:56:57.048243 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.048639 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048617    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 23:56:57.048825 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048809    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.048942 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.048922    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.049125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.049096    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xntdl\""
Jul 06 23:56:57.049321 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.049195    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 23:56:57.049321 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.049312    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 23:56:57.050970 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.050952    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.053351 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.053332    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.054862 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.054845    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 23:56:57.054975 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.054947    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6xgt2\""
Jul 06 23:56:57.055116 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.055098    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 23:56:57.055200 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.055187    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6sq55\""
Jul 06 23:56:57.055380 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.055366    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.055704 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.055688    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.055783 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.055764    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:56:57.058409 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.058393    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8jrcf\""
Jul 06 23:56:57.059097 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.059076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.063739 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.063724    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 23:56:57.064410 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.064388    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 23:56:57.064502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.064388    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 23:56:57.064502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.064465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p9hkp\""
Jul 06 23:56:57.076910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.076888    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 23:56:57.110606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.110574    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f46vc"
Jul 06 23:56:57.121647 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.121628    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f46vc"
Jul 06 23:56:57.131128 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.131110    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 23:56:57.142114 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-log-socket\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-ovn\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-netd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4hmr\" (UniqueName: \"kubernetes.io/projected/88bbc328-c0d6-415d-957a-b3d9a9c65aba-kube-api-access-x4hmr\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142180    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-conf\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-tmp\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142202    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-ovn\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142233 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrkzx\" (UniqueName: \"kubernetes.io/projected/579e3da3-6a24-4a7d-a719-519db369ad56-kube-api-access-hrkzx\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dfd3d92-996d-4407-990d-ac05269ea220-iptables-alerter-script\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142208    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-netd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142284    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-system-cni-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142346    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-system-cni-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf55712a-27f6-4122-9d81-349420350fb2-konnectivity-ca\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142356    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-conf\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142422    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-log-socket\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-bin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-etc-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-env-overrides\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-script-lib\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142540    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142559    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-etc-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.142570 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142571    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142572    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-tuned\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142705    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovn-node-metrics-cert\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-os-release\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142822    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysconfig\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142881    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysconfig\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.142973    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-env-overrides\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143114    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-os-release\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143142    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143277    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-script-lib\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143303    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32f0935-e944-44ed-a4c6-2f0e8c58376d-host\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143336    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.143399 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-systemd-units\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143404    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-systemd-units\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143412    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-sysctl-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143443    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143444    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579e3da3-6a24-4a7d-a719-519db369ad56-hosts-file\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143487    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29hbv\" (UniqueName: \"kubernetes.io/projected/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-kube-api-access-29hbv\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143509    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-run\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m44p\" (UniqueName: \"kubernetes.io/projected/0dfd3d92-996d-4407-990d-ac05269ea220-kube-api-access-9m44p\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143584    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143619    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-socket-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143624    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-run\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-registration-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-systemd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-kubernetes\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143731    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-device-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.144119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-daemon-config\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-bin\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143860    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-cni-bin\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143891    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-run-systemd\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143895    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cnibin\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143956    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-kubernetes\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.143957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzlhf\" (UniqueName: \"kubernetes.io/projected/636ea7c4-82a5-4e4c-bded-19c189eb703b-kube-api-access-fzlhf\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144035    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cnibin\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144061    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-os-release\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144090    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-multus\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144134    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-systemd\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144198    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4lws\" (UniqueName: \"kubernetes.io/projected/91a84a2f-d2eb-419f-b778-bab4fe49a581-kube-api-access-g4lws\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144200    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-systemd\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144252    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-sys-fs\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144310    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-socket-dir-parent\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144357    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r77fh\" (UniqueName: \"kubernetes.io/projected/d32f0935-e944-44ed-a4c6-2f0e8c58376d-kube-api-access-r77fh\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.144708 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-modprobe-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-sys\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf55712a-27f6-4122-9d81-349420350fb2-agent-certs\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144509    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-modprobe-d\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144512    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-node-log\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144548    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-kubelet\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144553    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-node-log\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-system-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144554    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-sys\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144612    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144633    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-kubelet\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-k8s-cni-cncf-io\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144696    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-netns\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-multus-certs\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk44r\" (UniqueName: \"kubernetes.io/projected/11671568-9dd3-45a0-825d-3b1a2c75234a-kube-api-access-qk44r\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-config\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144810    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-var-lib-kubelet\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145168 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144836    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-kubelet-config\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144867    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dfd3d92-996d-4407-990d-ac05269ea220-host-slash\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-slash\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144946    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-host\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144954    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-slash\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzdxl\" (UniqueName: \"kubernetes.io/projected/e83493b9-7ec7-4e8b-99f3-22cae75164ca-kube-api-access-xzdxl\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145006    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-kubelet\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-hostroot\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145100    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-netns\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-dbus\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145205    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-etc-kubernetes\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32f0935-e944-44ed-a4c6-2f0e8c58376d-serviceca\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145255    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-host\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145273    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-var-lib-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145390    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovnkube-config\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-lib-modules\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.145648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.144909    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-var-lib-kubelet\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145528    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e83493b9-7ec7-4e8b-99f3-22cae75164ca-lib-modules\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145525    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-var-lib-openvswitch\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145570    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e3da3-6a24-4a7d-a719-519db369ad56-tmp-dir\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145604    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-netns\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145643    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145665    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-cnibin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145685    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-cni-binary-copy\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145712    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-conf-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145784    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88bbc328-c0d6-415d-957a-b3d9a9c65aba-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.145828    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.146224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.146203    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-etc-tuned\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.146597 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.146276    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-ovn-node-metrics-cert\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.146873 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.146846    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e83493b9-7ec7-4e8b-99f3-22cae75164ca-tmp\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.151665 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.151645    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4hmr\" (UniqueName: \"kubernetes.io/projected/88bbc328-c0d6-415d-957a-b3d9a9c65aba-kube-api-access-x4hmr\") pod \"multus-additional-cni-plugins-5zdcz\" (UID: \"88bbc328-c0d6-415d-957a-b3d9a9c65aba\") " pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.152553 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.152508    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-29hbv\" (UniqueName: \"kubernetes.io/projected/a5755442-bf0b-4e4e-988b-34aeaf9c52fe-kube-api-access-29hbv\") pod \"ovnkube-node-x56f2\" (UID: \"a5755442-bf0b-4e4e-988b-34aeaf9c52fe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.157099 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.157064    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzdxl\" (UniqueName: \"kubernetes.io/projected/e83493b9-7ec7-4e8b-99f3-22cae75164ca-kube-api-access-xzdxl\") pod \"tuned-b9khx\" (UID: \"e83493b9-7ec7-4e8b-99f3-22cae75164ca\") " pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.246349 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246314    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m44p\" (UniqueName: \"kubernetes.io/projected/0dfd3d92-996d-4407-990d-ac05269ea220-kube-api-access-9m44p\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.246349 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246354    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-socket-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-registration-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246460    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-registration-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.246482    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-device-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246508    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-socket-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.246558    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:57.746529068 +0000 UTC m=+3.071532353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-daemon-config\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.246582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246578    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-device-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246602    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzlhf\" (UniqueName: \"kubernetes.io/projected/636ea7c4-82a5-4e4c-bded-19c189eb703b-kube-api-access-fzlhf\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246629    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-os-release\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-multus\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4lws\" (UniqueName: \"kubernetes.io/projected/91a84a2f-d2eb-419f-b778-bab4fe49a581-kube-api-access-g4lws\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246694    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-sys-fs\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246719    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-socket-dir-parent\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r77fh\" (UniqueName: \"kubernetes.io/projected/d32f0935-e944-44ed-a4c6-2f0e8c58376d-kube-api-access-r77fh\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246742    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-multus\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf55712a-27f6-4122-9d81-349420350fb2-agent-certs\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246759    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-os-release\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-system-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246809    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-k8s-cni-cncf-io\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-netns\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246847    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-multus-certs\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246858    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-socket-dir-parent\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk44r\" (UniqueName: \"kubernetes.io/projected/11671568-9dd3-45a0-825d-3b1a2c75234a-kube-api-access-qk44r\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247033 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-kubelet-config\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246897    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dfd3d92-996d-4407-990d-ac05269ea220-host-slash\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246898    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-k8s-cni-cncf-io\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246806    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-sys-fs\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246926    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-kubelet\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246965    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-hostroot\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246967    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-multus-certs\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.246994    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-dbus\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-etc-kubernetes\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247021    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-run-netns\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32f0935-e944-44ed-a4c6-2f0e8c58376d-serviceca\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247053    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-kubelet-config\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247061    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dfd3d92-996d-4407-990d-ac05269ea220-host-slash\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e3da3-6a24-4a7d-a719-519db369ad56-tmp-dir\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247088    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-daemon-config\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247112    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-system-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247141    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-cnibin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.247834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-cni-binary-copy\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247179    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-etc-kubernetes\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-conf-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247201    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-etc-selinux\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247213    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-conf-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247213    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-hostroot\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247223    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrkzx\" (UniqueName: \"kubernetes.io/projected/579e3da3-6a24-4a7d-a719-519db369ad56-kube-api-access-hrkzx\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dfd3d92-996d-4407-990d-ac05269ea220-iptables-alerter-script\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247275    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf55712a-27f6-4122-9d81-349420350fb2-konnectivity-ca\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247308    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/534cd6f5-0f18-4837-90e0-e83f0a16cefe-dbus\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-bin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247355    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-cni-bin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247144    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-host-var-lib-kubelet\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247406    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247441    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32f0935-e944-44ed-a4c6-2f0e8c58376d-host\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579e3da3-6a24-4a7d-a719-519db369ad56-hosts-file\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.248489 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247502    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/579e3da3-6a24-4a7d-a719-519db369ad56-tmp-dir\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247581    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/636ea7c4-82a5-4e4c-bded-19c189eb703b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247622    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d32f0935-e944-44ed-a4c6-2f0e8c58376d-host\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247654    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d32f0935-e944-44ed-a4c6-2f0e8c58376d-serviceca\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247670    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/579e3da3-6a24-4a7d-a719-519db369ad56-hosts-file\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247718    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-multus-cni-dir\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247251    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/11671568-9dd3-45a0-825d-3b1a2c75234a-cnibin\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.247746    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.247790    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:56:57.74777617 +0000 UTC m=+3.072779436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.247812    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0dfd3d92-996d-4407-990d-ac05269ea220-iptables-alerter-script\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.248360    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf55712a-27f6-4122-9d81-349420350fb2-konnectivity-ca\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.249034 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.248395    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/11671568-9dd3-45a0-825d-3b1a2c75234a-cni-binary-copy\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.249488 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.249469    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf55712a-27f6-4122-9d81-349420350fb2-agent-certs\") pod \"konnectivity-agent-bbcqn\" (UID: \"bf55712a-27f6-4122-9d81-349420350fb2\") " pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.258604 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.258583    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m44p\" (UniqueName: \"kubernetes.io/projected/0dfd3d92-996d-4407-990d-ac05269ea220-kube-api-access-9m44p\") pod \"iptables-alerter-k4rzb\" (UID: \"0dfd3d92-996d-4407-990d-ac05269ea220\") " pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.258862 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.258831    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzlhf\" (UniqueName: \"kubernetes.io/projected/636ea7c4-82a5-4e4c-bded-19c189eb703b-kube-api-access-fzlhf\") pod \"aws-ebs-csi-driver-node-nhxdv\" (UID: \"636ea7c4-82a5-4e4c-bded-19c189eb703b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.261276 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.261258    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk44r\" (UniqueName: \"kubernetes.io/projected/11671568-9dd3-45a0-825d-3b1a2c75234a-kube-api-access-qk44r\") pod \"multus-knjcv\" (UID: \"11671568-9dd3-45a0-825d-3b1a2c75234a\") " pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.261374 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.261338    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4lws\" (UniqueName: \"kubernetes.io/projected/91a84a2f-d2eb-419f-b778-bab4fe49a581-kube-api-access-g4lws\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.263106 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.263086    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:57.263189 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.263109    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:57.263189 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.263122    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:57.263189 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.263169    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:56:57.763159396 +0000 UTC m=+3.088162674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:57.266125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.266046    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r77fh\" (UniqueName: \"kubernetes.io/projected/d32f0935-e944-44ed-a4c6-2f0e8c58376d-kube-api-access-r77fh\") pod \"node-ca-c5v2z\" (UID: \"d32f0935-e944-44ed-a4c6-2f0e8c58376d\") " pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.266125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.266092    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrkzx\" (UniqueName: \"kubernetes.io/projected/579e3da3-6a24-4a7d-a719-519db369ad56-kube-api-access-hrkzx\") pod \"node-resolver-crvxn\" (UID: \"579e3da3-6a24-4a7d-a719-519db369ad56\") " pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.340154 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.340120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:56:57.348373 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.348352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zdcz"
Jul 06 23:56:57.354350 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.354325    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b9khx"
Jul 06 23:56:57.360038 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.360021    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-crvxn"
Jul 06 23:56:57.366764 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.366747    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4rzb"
Jul 06 23:56:57.373375 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.373350    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:56:57.380275 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.380258    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv"
Jul 06 23:56:57.386125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.386101    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-knjcv"
Jul 06 23:56:57.390929 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.390910    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c5v2z"
Jul 06 23:56:57.751367 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.751341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:57.751502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.751387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:57.751502 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.751480    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:57.751502 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.751501    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:57.751611 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.751540    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:56:58.751525075 +0000 UTC m=+4.076528337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:57.751611 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.751554    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:56:58.751547635 +0000 UTC m=+4.076550897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:57.802350 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.802327    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 23:56:57.815230 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.815200    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11671568_9dd3_45a0_825d_3b1a2c75234a.slice/crio-64c8228eb18061150272dc43d12430eb7b30edd800da28833b41fba27c6e9b91 WatchSource:0}: Error finding container 64c8228eb18061150272dc43d12430eb7b30edd800da28833b41fba27c6e9b91: Status 404 returned error can't find the container with id 64c8228eb18061150272dc43d12430eb7b30edd800da28833b41fba27c6e9b91
Jul 06 23:56:57.816648 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.816620    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88bbc328_c0d6_415d_957a_b3d9a9c65aba.slice/crio-5b044877aa7a60c9723a23f8a8830036f3567c264cb231626c08f7f2c81c885f WatchSource:0}: Error finding container 5b044877aa7a60c9723a23f8a8830036f3567c264cb231626c08f7f2c81c885f: Status 404 returned error can't find the container with id 5b044877aa7a60c9723a23f8a8830036f3567c264cb231626c08f7f2c81c885f
Jul 06 23:56:57.817958 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.817935    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dfd3d92_996d_4407_990d_ac05269ea220.slice/crio-217242261b16142168335e6cd98440e6889861929bd27d81bca26ce93ad109f5 WatchSource:0}: Error finding container 217242261b16142168335e6cd98440e6889861929bd27d81bca26ce93ad109f5: Status 404 returned error can't find the container with id 217242261b16142168335e6cd98440e6889861929bd27d81bca26ce93ad109f5
Jul 06 23:56:57.818906 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.818880    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636ea7c4_82a5_4e4c_bded_19c189eb703b.slice/crio-06ea3d51e2953fb64455f7dca9a96f04ab8f6cfcc9ab76699aff16de4f629b3c WatchSource:0}: Error finding container 06ea3d51e2953fb64455f7dca9a96f04ab8f6cfcc9ab76699aff16de4f629b3c: Status 404 returned error can't find the container with id 06ea3d51e2953fb64455f7dca9a96f04ab8f6cfcc9ab76699aff16de4f629b3c
Jul 06 23:56:57.821959 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.821936    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf55712a_27f6_4122_9d81_349420350fb2.slice/crio-ce0162f5eeef666d1e2553a2b418efb124dc5705c8ff51f895a6eb7dcf81d5d6 WatchSource:0}: Error finding container ce0162f5eeef666d1e2553a2b418efb124dc5705c8ff51f895a6eb7dcf81d5d6: Status 404 returned error can't find the container with id ce0162f5eeef666d1e2553a2b418efb124dc5705c8ff51f895a6eb7dcf81d5d6
Jul 06 23:56:57.822831 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.822752    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579e3da3_6a24_4a7d_a719_519db369ad56.slice/crio-19d0b2ba6b9d6feb54a7bc5054e4b9800cc3e0e96fddde51bf117dd3b527f579 WatchSource:0}: Error finding container 19d0b2ba6b9d6feb54a7bc5054e4b9800cc3e0e96fddde51bf117dd3b527f579: Status 404 returned error can't find the container with id 19d0b2ba6b9d6feb54a7bc5054e4b9800cc3e0e96fddde51bf117dd3b527f579
Jul 06 23:56:57.823640 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.823620    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83493b9_7ec7_4e8b_99f3_22cae75164ca.slice/crio-f5d70b7e1c3db3b2b6ba932fd131d35d2cb892304046dc6e6c8353514f13b636 WatchSource:0}: Error finding container f5d70b7e1c3db3b2b6ba932fd131d35d2cb892304046dc6e6c8353514f13b636: Status 404 returned error can't find the container with id f5d70b7e1c3db3b2b6ba932fd131d35d2cb892304046dc6e6c8353514f13b636
Jul 06 23:56:57.824943 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.824906    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5755442_bf0b_4e4e_988b_34aeaf9c52fe.slice/crio-39f9848af68bfdfd1376f985941a93f399cf987895dd55d8cb583b74dd81e577 WatchSource:0}: Error finding container 39f9848af68bfdfd1376f985941a93f399cf987895dd55d8cb583b74dd81e577: Status 404 returned error can't find the container with id 39f9848af68bfdfd1376f985941a93f399cf987895dd55d8cb583b74dd81e577
Jul 06 23:56:57.829183 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:56:57.829045    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd32f0935_e944_44ed_a4c6_2f0e8c58376d.slice/crio-4c832ad7d49eb2b0f2853eec820a8ba19636e7aa0bebb82f1ca04be5432ea45d WatchSource:0}: Error finding container 4c832ad7d49eb2b0f2853eec820a8ba19636e7aa0bebb82f1ca04be5432ea45d: Status 404 returned error can't find the container with id 4c832ad7d49eb2b0f2853eec820a8ba19636e7aa0bebb82f1ca04be5432ea45d
Jul 06 23:56:57.851678 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:57.851656    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:57.851777 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.851768    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:57.851901 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.851782    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:57.851901 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.851790    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:57.851901 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:57.851830    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:56:58.851818276 +0000 UTC m=+4.176821538 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:58.125009 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.124794    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:57 +0000 UTC" deadline="2028-01-11 06:54:55.934431758 +0000 UTC"
Jul 06 23:56:58.125009 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.125006    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13278h57m57.809428501s"
Jul 06 23:56:58.152205 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.152179    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:58.152328 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.152282    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:56:58.161177 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.161151    2583 generic.go:356] "Generic (PLEG): container finished" podID="15ea2d76c80edf5184bf6ff447bd6336" containerID="c360a31aae755841bcf22a160f0719df42aeb06f80d26aa4ff6c1c2d6275bf90" exitCode=0
Jul 06 23:56:58.161284 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.161234    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal" event={"ID":"15ea2d76c80edf5184bf6ff447bd6336","Type":"ContainerDied","Data":"c360a31aae755841bcf22a160f0719df42aeb06f80d26aa4ff6c1c2d6275bf90"}
Jul 06 23:56:58.162701 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.162672    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal" event={"ID":"936b88c7565d9c88f04260646b598913","Type":"ContainerStarted","Data":"224913e56f4b0c2a21a0c956702f8e8359b50f04202fc2b8e096bf20c4bde069"}
Jul 06 23:56:58.163811 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.163754    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-crvxn" event={"ID":"579e3da3-6a24-4a7d-a719-519db369ad56","Type":"ContainerStarted","Data":"19d0b2ba6b9d6feb54a7bc5054e4b9800cc3e0e96fddde51bf117dd3b527f579"}
Jul 06 23:56:58.167666 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.167643    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c5v2z" event={"ID":"d32f0935-e944-44ed-a4c6-2f0e8c58376d","Type":"ContainerStarted","Data":"4c832ad7d49eb2b0f2853eec820a8ba19636e7aa0bebb82f1ca04be5432ea45d"}
Jul 06 23:56:58.168918 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.168892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b9khx" event={"ID":"e83493b9-7ec7-4e8b-99f3-22cae75164ca","Type":"ContainerStarted","Data":"f5d70b7e1c3db3b2b6ba932fd131d35d2cb892304046dc6e6c8353514f13b636"}
Jul 06 23:56:58.170263 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.170237    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"39f9848af68bfdfd1376f985941a93f399cf987895dd55d8cb583b74dd81e577"}
Jul 06 23:56:58.171683 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.171661    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbcqn" event={"ID":"bf55712a-27f6-4122-9d81-349420350fb2","Type":"ContainerStarted","Data":"ce0162f5eeef666d1e2553a2b418efb124dc5705c8ff51f895a6eb7dcf81d5d6"}
Jul 06 23:56:58.173096 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.173029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-knjcv" event={"ID":"11671568-9dd3-45a0-825d-3b1a2c75234a","Type":"ContainerStarted","Data":"64c8228eb18061150272dc43d12430eb7b30edd800da28833b41fba27c6e9b91"}
Jul 06 23:56:58.174959 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.174935    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv" event={"ID":"636ea7c4-82a5-4e4c-bded-19c189eb703b","Type":"ContainerStarted","Data":"06ea3d51e2953fb64455f7dca9a96f04ab8f6cfcc9ab76699aff16de4f629b3c"}
Jul 06 23:56:58.176394 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.176370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4rzb" event={"ID":"0dfd3d92-996d-4407-990d-ac05269ea220","Type":"ContainerStarted","Data":"217242261b16142168335e6cd98440e6889861929bd27d81bca26ce93ad109f5"}
Jul 06 23:56:58.178610 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.178575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerStarted","Data":"5b044877aa7a60c9723a23f8a8830036f3567c264cb231626c08f7f2c81c885f"}
Jul 06 23:56:58.200614 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.200559    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-92.ec2.internal" podStartSLOduration=2.200509171 podStartE2EDuration="2.200509171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:58.199399422 +0000 UTC m=+3.524402701" watchObservedRunningTime="2026-07-06 23:56:58.200509171 +0000 UTC m=+3.525512456"
Jul 06 23:56:58.759713 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.759623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:58.759895 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.759804    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:58.759895 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.759874    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:00.759855398 +0000 UTC m=+6.084858674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:56:58.760269 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.760049    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:58.760269 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.760187    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:58.760269 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.760247    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:57:00.760231415 +0000 UTC m=+6.085234679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:56:58.860912 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:58.860837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:56:58.861101 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.861035    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:56:58.861101 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.861057    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:56:58.861101 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.861072    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:58.861266 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:58.861130    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:00.861113152 +0000 UTC m=+6.186116429 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:56:59.125836 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.125789    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:51:57 +0000 UTC" deadline="2028-01-24 09:50:55.508907873 +0000 UTC"
Jul 06 23:56:59.125836 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.125830    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13593h53m56.383082085s"
Jul 06 23:56:59.153440 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.153408    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:56:59.153599 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:59.153549    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:56:59.155619 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.155481    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:56:59.155619 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:56:59.155584    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:56:59.201533 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.201501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal" event={"ID":"15ea2d76c80edf5184bf6ff447bd6336","Type":"ContainerStarted","Data":"e79076fbbb010983e172502a69656432f468404522ec33bde5fb8cd32f4f932c"}
Jul 06 23:56:59.219717 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:56:59.219661    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-92.ec2.internal" podStartSLOduration=3.219639335 podStartE2EDuration="3.219639335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:56:59.219532227 +0000 UTC m=+4.544535515" watchObservedRunningTime="2026-07-06 23:56:59.219639335 +0000 UTC m=+4.544642620"
Jul 06 23:57:00.152729 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:00.152187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:00.152729 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.152352    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:00.778287 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:00.778221    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:00.778524 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:00.778319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:00.778524 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.778470    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:00.778638 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.778536    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:04.778516705 +0000 UTC m=+10.103519981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:00.778942 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.778920    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:00.779009 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.778976    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:57:04.778961875 +0000 UTC m=+10.103965137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:00.879634 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:00.879595    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:00.879817 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.879801    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:57:00.879875 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.879819    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:57:00.879875 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.879835    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:00.879971 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:00.879891    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:04.879872993 +0000 UTC m=+10.204876278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:01.152429 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:01.152398    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:01.152613 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:01.152531    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:01.152613 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:01.152601    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:01.152742 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:01.152722    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:02.152350 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:02.152276    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:02.152727 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:02.152473    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:03.152636 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:03.152601    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:03.153235 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:03.152601    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:03.153235 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:03.152747    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:03.153235 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:03.152874    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:04.152039 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:04.152005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:04.152223 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.152134    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:04.815515    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:04.815609    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.815669    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.815736    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.815753    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.815731686 +0000 UTC m=+18.140734961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:04.815814 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.815789    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.815774757 +0000 UTC m=+18.140778024 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:04.916812 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:04.916773    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:04.916962 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.916947    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:57:04.917028 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.916975    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:57:04.917028 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.916989    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:04.917131 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:04.917050    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:12.917031868 +0000 UTC m=+18.242035148 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:05.155908 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:05.154964    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:05.155908 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:05.155139    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:05.155908 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:05.155732    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:05.155908 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:05.155854    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:06.152899 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:06.152750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:06.152899 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:06.152887    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:07.152652 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:07.152621    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:07.152834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:07.152653    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:07.152834 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:07.152772    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:07.152947 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:07.152914    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:08.152242 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:08.152212    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:08.152408 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:08.152380    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:09.152093 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:09.152059    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:09.152564 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:09.152172    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:09.152564 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:09.152232    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:09.152564 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:09.152309    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:10.152650 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:10.152623    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:10.153086 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:10.152739    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:11.152569 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:11.152535    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:11.152760 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:11.152535    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:11.152760 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:11.152655    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:11.153208 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:11.152784    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:12.152817 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:12.152782    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:12.153245 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.152896    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:12.880993 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:12.880951    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:12.881191 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:12.881040    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:12.881191 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.881129    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:12.881191 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.881143    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:12.881369 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.881197    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:57:28.881180715 +0000 UTC m=+34.206183994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:12.881369 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.881216    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:28.881207112 +0000 UTC m=+34.206210376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:12.981507 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:12.981468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:12.981740 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.981645    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:57:12.981740 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.981667    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:57:12.981740 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.981680    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:12.981877 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:12.981747    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:28.981729969 +0000 UTC m=+34.306733235 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:13.153072 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:13.152984    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:13.153528 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:13.153124    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:13.153528 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:13.153187    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:13.153528 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:13.153337    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:14.152729 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:14.152689    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:14.152890 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:14.152816    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:15.153273 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:15.153235    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:15.153765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:15.153370    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:15.153765 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:15.153414    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:15.153765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:15.153521    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:16.152651 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:16.152627    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:16.152785 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:16.152719    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:17.153139 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.152774    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:17.153957 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.152781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:17.153957 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:17.153246    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:17.153957 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:17.153317    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:17.238188 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.238111    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-crvxn" event={"ID":"579e3da3-6a24-4a7d-a719-519db369ad56","Type":"ContainerStarted","Data":"1f3bcaac23379189608832142bc3109b0480d61f58bbf496c16fe2f24aca00cd"}
Jul 06 23:57:17.239665 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.239638    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c5v2z" event={"ID":"d32f0935-e944-44ed-a4c6-2f0e8c58376d","Type":"ContainerStarted","Data":"f1b8d2bb8588c2b2847e68206c6b266109092d44237810796779b1faa43dd2fc"}
Jul 06 23:57:17.240988 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.240969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b9khx" event={"ID":"e83493b9-7ec7-4e8b-99f3-22cae75164ca","Type":"ContainerStarted","Data":"1d193520bfa68e4bf0d35ec7c6e8f22957b69f2e87fc9818d2585322baaefeb7"}
Jul 06 23:57:17.243867 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.243840    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"dff621228214b8dc02d80a2796355a1a055be94cd8781882131b9bb5f77e6826"}
Jul 06 23:57:17.243975 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.243912    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"66b956960e8f37183097738bd28652a4149d03e9a053cc0630589766db4f65ec"}
Jul 06 23:57:17.243975 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.243956    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"191accf3c422c7bb7a1626f071351ca98354c8b1f874bc56e4449e1a2833dcd8"}
Jul 06 23:57:17.244083 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.244008    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"cf63195a7903f33ddafffa38feb2c2d4615bb42276235b99183f8ebd45e72d3c"}
Jul 06 23:57:17.244083 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.244053    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"831931e87c3ff96b42b856775939dca02a9ab93872496d11c259127944470487"}
Jul 06 23:57:17.244181 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.244096    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"0acd83cf689f619b43b28336002622946295f75065262a875ec2aaa24e8d84b3"}
Jul 06 23:57:17.245435 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.245411    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bbcqn" event={"ID":"bf55712a-27f6-4122-9d81-349420350fb2","Type":"ContainerStarted","Data":"70134a14371efabe1e8501f793ec91e568d8e3f679894fa2773f0dd5c83f4fa1"}
Jul 06 23:57:17.246815 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.246788    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-knjcv" event={"ID":"11671568-9dd3-45a0-825d-3b1a2c75234a","Type":"ContainerStarted","Data":"555e6af7069a3044fcc9fa69f1955c59ae83b649491f7590b2ff94a3eb3d32e8"}
Jul 06 23:57:17.248100 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.248081    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv" event={"ID":"636ea7c4-82a5-4e4c-bded-19c189eb703b","Type":"ContainerStarted","Data":"acffdd5e4484d5150d9d5d47e25f3eb178f11936ee1ab1cd5df7a6884c29d3a5"}
Jul 06 23:57:17.249517 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.249499    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="d6dce52694a1f32ee9f78b827f84d933000c16a087988649fcc0c5f721bface6" exitCode=0
Jul 06 23:57:17.249610 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.249524    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"d6dce52694a1f32ee9f78b827f84d933000c16a087988649fcc0c5f721bface6"}
Jul 06 23:57:17.258341 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.258276    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-crvxn" podStartSLOduration=3.810631855 podStartE2EDuration="22.258258102s" totalImagesPullingTime="18.447626247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.257840593 +0000 UTC m=+22.582843888" watchObservedRunningTime="2026-07-06 23:57:17.258258102 +0000 UTC m=+22.583261389"
Jul 06 23:57:17.282093 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.282043    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c5v2z" podStartSLOduration=3.843769769 podStartE2EDuration="22.282028371s" totalImagesPullingTime="18.438258602s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.281783337 +0000 UTC m=+22.606786622" watchObservedRunningTime="2026-07-06 23:57:17.282028371 +0000 UTC m=+22.607031655"
Jul 06 23:57:17.332932 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.332869    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b9khx" podStartSLOduration=3.886849486 podStartE2EDuration="22.332852985s" totalImagesPullingTime="18.446003499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.332718941 +0000 UTC m=+22.657722225" watchObservedRunningTime="2026-07-06 23:57:17.332852985 +0000 UTC m=+22.657856268"
Jul 06 23:57:17.350987 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.350928    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bbcqn" podStartSLOduration=12.034085032 podStartE2EDuration="22.350912607s" totalImagesPullingTime="10.316827575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.350422338 +0000 UTC m=+22.675425624" watchObservedRunningTime="2026-07-06 23:57:17.350912607 +0000 UTC m=+22.675915891"
Jul 06 23:57:17.373266 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.373201    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-knjcv" podStartSLOduration=3.858493238 podStartE2EDuration="22.373182032s" totalImagesPullingTime="18.514688794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:17.372847263 +0000 UTC m=+22.697850550" watchObservedRunningTime="2026-07-06 23:57:17.373182032 +0000 UTC m=+22.698185313"
Jul 06 23:57:17.894897 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:17.894858    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 23:57:18.093777 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.093559    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:57:17.89488718Z","UUID":"490e9e1d-90fd-41e7-bec7-b206bd90078c","Handler":null,"Name":"","Endpoint":""}
Jul 06 23:57:18.095120 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.095093    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 23:57:18.095242 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.095136    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 23:57:18.152666 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.152639    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:18.152816 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:18.152758    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:18.254314 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.254261    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv" event={"ID":"636ea7c4-82a5-4e4c-bded-19c189eb703b","Type":"ContainerStarted","Data":"1c9ea65bfa88e76853e071f5914304961e2b2b40b72c77e1c88f41bafac3e3be"}
Jul 06 23:57:18.255824 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.255793    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4rzb" event={"ID":"0dfd3d92-996d-4407-990d-ac05269ea220","Type":"ContainerStarted","Data":"6facc59d6bc5d980d6a3855f6de68d42d8112286c7ee32fb64e2bd3ed862677a"}
Jul 06 23:57:18.290485 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:18.290428    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4rzb" podStartSLOduration=4.841300329 podStartE2EDuration="23.290405968s" totalImagesPullingTime="18.449105639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:18.290351138 +0000 UTC m=+23.615354422" watchObservedRunningTime="2026-07-06 23:57:18.290405968 +0000 UTC m=+23.615409252"
Jul 06 23:57:19.152560 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:19.152539    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:19.152683 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:19.152644    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:19.152757 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:19.152739    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:19.152881 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:19.152856    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:19.260874 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:19.260620    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv" event={"ID":"636ea7c4-82a5-4e4c-bded-19c189eb703b","Type":"ContainerStarted","Data":"be39d5d98600c8836816b0c997fc371311a098a8106d30294ef68a4f6fb8eb9f"}
Jul 06 23:57:19.265801 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:19.265746    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"06facf9d6d21ca9eba3753efdaa2580b59ccf6273ad014847eeadb652dd3a0e6"}
Jul 06 23:57:20.152815 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:20.152781    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:20.153003 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:20.152892    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:20.707993 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:20.707955    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:57:20.708606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:20.708574    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:57:20.765495 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:20.765446    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nhxdv" podStartSLOduration=5.092989012 podStartE2EDuration="25.765430253s" totalImagesPullingTime="20.672441241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:19.288634758 +0000 UTC m=+24.613638041" watchObservedRunningTime="2026-07-06 23:57:20.765430253 +0000 UTC m=+26.090433562"
Jul 06 23:57:21.152966 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:21.152893    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:21.153159 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:21.153033    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:21.153159 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:21.153091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:21.153303 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:21.153216    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:21.270959 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:21.270926    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:57:21.271497 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:21.271474    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bbcqn"
Jul 06 23:57:22.152025 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.151844    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:22.152667 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:22.152090    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:22.275886 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.275844    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" event={"ID":"a5755442-bf0b-4e4e-988b-34aeaf9c52fe","Type":"ContainerStarted","Data":"209d19dc273bc59747a13d89cd3e998eaa691044b3b7f7c1e5beff6459f61465"}
Jul 06 23:57:22.276170 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.276137    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:22.276311 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.276213    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:22.277701 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.277678    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="031be39484e25a314cf578ba049ca964abf591ec8b69648ba60b4acbce7078a3" exitCode=0
Jul 06 23:57:22.277799 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.277767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"031be39484e25a314cf578ba049ca964abf591ec8b69648ba60b4acbce7078a3"}
Jul 06 23:57:22.292155 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.292132    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:22.309345 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:22.309283    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2" podStartSLOduration=8.78916068 podStartE2EDuration="27.309268623s" totalImagesPullingTime="18.520107943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:22.307959122 +0000 UTC m=+27.632962408" watchObservedRunningTime="2026-07-06 23:57:22.309268623 +0000 UTC m=+27.634271906"
Jul 06 23:57:23.152531 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.152451    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:23.153012 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:23.152575    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:23.153012 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.152631    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:23.153012 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:23.152730    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:23.280183 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.280155    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:23.294759 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.294736    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:23.624224 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.624188    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hxjb2"]
Jul 06 23:57:23.624457 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.624322    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:23.624457 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:23.624433    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:23.647532 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.647504    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxpmb"]
Jul 06 23:57:23.647663 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.647611    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:23.647720 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:23.647701    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:23.670929 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.670856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6rtl"]
Jul 06 23:57:23.671057 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:23.670944    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:23.671103 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:23.671054    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:24.283662 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:24.283626    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="df004bb5ae856acd949c1a7f948bce67a584e10c34a54bfa64f12a493de613a1" exitCode=0
Jul 06 23:57:24.284070 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:24.283707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"df004bb5ae856acd949c1a7f948bce67a584e10c34a54bfa64f12a493de613a1"}
Jul 06 23:57:25.154081 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:25.153868    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:25.154195 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:25.153946    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:25.154195 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:25.154167    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:25.154316 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:25.153985    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:25.154316 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:25.154244    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:25.154414 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:25.154347    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:25.288348 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:25.288321    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerStarted","Data":"26e08bfe10727fbbb785084c7c8b76a25888c93b747f988ad1f8032532930281"}
Jul 06 23:57:26.293121 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:26.293089    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="26e08bfe10727fbbb785084c7c8b76a25888c93b747f988ad1f8032532930281" exitCode=0
Jul 06 23:57:26.293699 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:26.293131    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"26e08bfe10727fbbb785084c7c8b76a25888c93b747f988ad1f8032532930281"}
Jul 06 23:57:27.152515 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:27.152479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:27.152659 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:27.152479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:27.152659 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:27.152640    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j6rtl" podUID="91a84a2f-d2eb-419f-b778-bab4fe49a581"
Jul 06 23:57:27.152780 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:27.152479    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:27.152780 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:27.152685    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fxpmb" podUID="e7da8523-bde6-4880-8edd-5bc6f5001d4d"
Jul 06 23:57:27.152867 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:27.152791    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hxjb2" podUID="534cd6f5-0f18-4837-90e0-e83f0a16cefe"
Jul 06 23:57:28.903260 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.903220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:28.903944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.903307    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:28.903944 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:28.903406    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:28.903944 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:28.903427    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:28.903944 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:28.903487    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret podName:534cd6f5-0f18-4837-90e0-e83f0a16cefe nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.903467008 +0000 UTC m=+66.228470284 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret") pod "global-pull-secret-syncer-hxjb2" (UID: "534cd6f5-0f18-4837-90e0-e83f0a16cefe") : object "kube-system"/"original-pull-secret" not registered
Jul 06 23:57:28.903944 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:28.903511    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.903500368 +0000 UTC m=+66.228503643 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 23:57:28.948910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.948877    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-92.ec2.internal" event="NodeReady"
Jul 06 23:57:28.949075 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.949017    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 23:57:28.983557 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.983523    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:57:28.986137 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.986113    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:28.988664 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.988637    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 23:57:28.988839 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.988816    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 23:57:28.988951 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.988858    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 23:57:28.988951 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.988901    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qtrrj\""
Jul 06 23:57:28.998222 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.998194    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 23:57:28.998908 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:28.998879    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:57:29.001378 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.001357    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qrhjm"]
Jul 06 23:57:29.003382 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.003360    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ccxln"]
Jul 06 23:57:29.003517 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.003498    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.003700 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.003679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:29.003939 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.003828    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 23:57:29.003939 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.003850    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 23:57:29.003939 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.003864    2583 projected.go:196] Error preparing data for projected volume kube-api-access-vjkc6 for pod openshift-network-diagnostics/network-check-target-fxpmb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:29.003939 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.003915    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6 podName:e7da8523-bde6-4880-8edd-5bc6f5001d4d nodeName:}" failed. No retries permitted until 2026-07-06 23:58:01.003903019 +0000 UTC m=+66.328906286 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vjkc6" (UniqueName: "kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6") pod "network-check-target-fxpmb" (UID: "e7da8523-bde6-4880-8edd-5bc6f5001d4d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 23:57:29.005143 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.005127    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.005797 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.005781    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 23:57:29.005889 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.005799    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s7rq7\""
Jul 06 23:57:29.006121 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.006103    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 23:57:29.007783 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.007758    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 23:57:29.007915 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.007899    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 23:57:29.007963 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.007950    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 23:57:29.008194 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.008179    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2f46f\""
Jul 06 23:57:29.016419 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.016398    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ccxln"]
Jul 06 23:57:29.017232 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.017208    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qrhjm"]
Jul 06 23:57:29.104963 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.104879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.104963 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.104919    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.104963 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.104944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9745e43d-f62d-476a-980f-54ad7ccf8c8f-config-volume\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.104990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2f224ba-1efd-4b24-b548-44b7c311c4f3-cert\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105063    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwqwv\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105103    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105140    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105177    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105244 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105241    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.105496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9745e43d-f62d-476a-980f-54ad7ccf8c8f-tmp-dir\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.105496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m4js\" (UniqueName: \"kubernetes.io/projected/9745e43d-f62d-476a-980f-54ad7ccf8c8f-kube-api-access-5m4js\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.105496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105376    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.105496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.105391    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2v7z\" (UniqueName: \"kubernetes.io/projected/e2f224ba-1efd-4b24-b548-44b7c311c4f3-kube-api-access-n2v7z\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.152009 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.151975    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:57:29.152009 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.152002    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:57:29.152218 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.151973    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154837    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6xkb\""
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154849    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154870    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdx8b\""
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154896    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154840    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:57:29.155046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.154843    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:57:29.156239 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.156192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:29.205920 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.205881    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206113 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.205928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206113 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.206113 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9745e43d-f62d-476a-980f-54ad7ccf8c8f-tmp-dir\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.206358 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.206199    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:29.206358 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.206307    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:57:29.706272691 +0000 UTC m=+35.031275969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:29.206473 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m4js\" (UniqueName: \"kubernetes.io/projected/9745e43d-f62d-476a-980f-54ad7ccf8c8f-kube-api-access-5m4js\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.206473 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206402    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9745e43d-f62d-476a-980f-54ad7ccf8c8f-tmp-dir\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.206473 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206410    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206473 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206436    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2v7z\" (UniqueName: \"kubernetes.io/projected/e2f224ba-1efd-4b24-b548-44b7c311c4f3-kube-api-access-n2v7z\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.206473 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206468    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206711 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206711 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9745e43d-f62d-476a-980f-54ad7ccf8c8f-config-volume\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.206711 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206542    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2f224ba-1efd-4b24-b548-44b7c311c4f3-cert\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.206711 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206576    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwqwv\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206711 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.206941 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.206829    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.207352 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.207100    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.207352 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.207130    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9745e43d-f62d-476a-980f-54ad7ccf8c8f-config-volume\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.207352 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.207235    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:29.207352 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.207251    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:29.207352 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.207324    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:29.707285257 +0000 UTC m=+35.032288532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:29.207731 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.207396    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.207910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.207889    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.211153 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.211129    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.211259 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.211151    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e2f224ba-1efd-4b24-b548-44b7c311c4f3-cert\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.211259 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.211129    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.218220 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.218169    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m4js\" (UniqueName: \"kubernetes.io/projected/9745e43d-f62d-476a-980f-54ad7ccf8c8f-kube-api-access-5m4js\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.218549 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.218524    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2v7z\" (UniqueName: \"kubernetes.io/projected/e2f224ba-1efd-4b24-b548-44b7c311c4f3-kube-api-access-n2v7z\") pod \"ingress-canary-ccxln\" (UID: \"e2f224ba-1efd-4b24-b548-44b7c311c4f3\") " pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.218842 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.218817    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.219056 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.219036    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwqwv\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.250653 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.250607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:29.321276 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.321236    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ccxln"
Jul 06 23:57:29.711005 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.710970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:29.711209 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:29.711016    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:29.711209 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.711152    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:29.711209 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.711174    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:29.711209 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.711197    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:29.711431 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.711235    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:30.711216003 +0000 UTC m=+36.036219279 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:29.711431 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:29.711254    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:57:30.711245151 +0000 UTC m=+36.036248434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:30.272776 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:30.272621    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ccxln"]
Jul 06 23:57:30.276326 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:57:30.276274    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f224ba_1efd_4b24_b548_44b7c311c4f3.slice/crio-24bb2c8ba51dd5cfd59d800f45425807ae563e5c022411b2e2e076b1748e87e6 WatchSource:0}: Error finding container 24bb2c8ba51dd5cfd59d800f45425807ae563e5c022411b2e2e076b1748e87e6: Status 404 returned error can't find the container with id 24bb2c8ba51dd5cfd59d800f45425807ae563e5c022411b2e2e076b1748e87e6
Jul 06 23:57:30.305481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:30.305440    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerStarted","Data":"cfe1f6b15a2496a5ae43ffe632900fcd01f47734d79f213fe010eed0a59fe40c"}
Jul 06 23:57:30.306498 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:30.306478    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ccxln" event={"ID":"e2f224ba-1efd-4b24-b548-44b7c311c4f3","Type":"ContainerStarted","Data":"24bb2c8ba51dd5cfd59d800f45425807ae563e5c022411b2e2e076b1748e87e6"}
Jul 06 23:57:30.718550 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:30.718501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:30.718746 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:30.718561    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:30.718746 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:30.718680    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:30.718746 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:30.718681    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:30.718746 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:30.718707    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:30.718746 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:30.718745    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:57:32.718726444 +0000 UTC m=+38.043729723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:30.718997 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:30.718777    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:32.718759099 +0000 UTC m=+38.043762367 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:31.311747 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:31.311710    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="cfe1f6b15a2496a5ae43ffe632900fcd01f47734d79f213fe010eed0a59fe40c" exitCode=0
Jul 06 23:57:31.312128 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:31.311754    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"cfe1f6b15a2496a5ae43ffe632900fcd01f47734d79f213fe010eed0a59fe40c"}
Jul 06 23:57:32.317514 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:32.317483    2583 generic.go:356] "Generic (PLEG): container finished" podID="88bbc328-c0d6-415d-957a-b3d9a9c65aba" containerID="9b19140b3b44d5d163434ad6b9f5103573bbf14f43db45c4b80b3680e05cf81f" exitCode=0
Jul 06 23:57:32.318081 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:32.317541    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerDied","Data":"9b19140b3b44d5d163434ad6b9f5103573bbf14f43db45c4b80b3680e05cf81f"}
Jul 06 23:57:32.734734 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:32.734539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:32.734890 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:32.734752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:32.734890 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:32.734702    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:32.734890 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:32.734825    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:32.734890 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:32.734883    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:36.734863864 +0000 UTC m=+42.059867129 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:32.735052 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:32.734900    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:32.735052 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:32.734940    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:57:36.73492705 +0000 UTC m=+42.059930311 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:33.323125 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.323092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" event={"ID":"88bbc328-c0d6-415d-957a-b3d9a9c65aba","Type":"ContainerStarted","Data":"a5eb9e588998582f2a5c56aa59c2a06beb4ef4fbe036d9f4adad65d26eb7f8b8"}
Jul 06 23:57:33.324512 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.324491    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ccxln" event={"ID":"e2f224ba-1efd-4b24-b548-44b7c311c4f3","Type":"ContainerStarted","Data":"db4388a577743ac375f758fc03d6bc05bed3a67079a1cdbb2f4d45d997fd368e"}
Jul 06 23:57:33.377359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.377284    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ccxln" podStartSLOduration=3.086245612 podStartE2EDuration="5.377266838s" totalImagesPullingTime="2.291021226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:33.377174535 +0000 UTC m=+38.702177820" watchObservedRunningTime="2026-07-06 23:57:33.377266838 +0000 UTC m=+38.702270122"
Jul 06 23:57:33.377532 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.377425    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5zdcz" podStartSLOduration=9.897485947 podStartE2EDuration="38.377418935s" totalImagesPullingTime="28.479932988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:33.355922768 +0000 UTC m=+38.680926053" watchObservedRunningTime="2026-07-06 23:57:33.377418935 +0000 UTC m=+38.702422236"
Jul 06 23:57:33.392776 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.392743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:33.494353 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.494269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:33.598888 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:33.598849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:34.947872 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:34.947837    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:35.049430 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:35.049399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:35.968013 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:35.967981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:36.067114 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:36.067079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:36.763546 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:36.763497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:36.763546 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:36.763550    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:36.763765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:36.763661    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:36.763765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:36.763665    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:36.763765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:36.763681    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:36.763765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:36.763730    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.7637106 +0000 UTC m=+50.088713865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:36.763765 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:36.763743    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.763737509 +0000 UTC m=+50.088740772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:38.030134 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:38.030100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.129424 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:38.129393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.229521 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:38.229491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.849120 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:38.849077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:38.945391 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:38.945353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:39.197706 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:39.197668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:39.300458 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:39.300427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:41.369615 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:41.369578    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:44.219165 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.219088    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"]
Jul 06 23:57:44.263273 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.263241    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"]
Jul 06 23:57:44.263445 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.263360    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.272754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.269228    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 06 23:57:44.272754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.268476    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.272754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.269532    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rt8s4\""
Jul 06 23:57:44.272754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.269614    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.272754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.269985    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 06 23:57:44.318753 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.318719    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.318910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.318769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpq7v\" (UniqueName: \"kubernetes.io/projected/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-kube-api-access-cpq7v\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.318910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.318799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.321359 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.321332    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ztxlq"]
Jul 06 23:57:44.344612 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.344581    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"]
Jul 06 23:57:44.344750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.344708    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.347591 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.347566    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 06 23:57:44.347713 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.347570    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.347713 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.347571    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 06 23:57:44.347713 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.347692    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qwfjl\""
Jul 06 23:57:44.347884 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.347718    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.352242 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.352218    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 06 23:57:44.360557 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.360483    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"]
Jul 06 23:57:44.360650 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.360558    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"
Jul 06 23:57:44.363065 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.363044    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.363172 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.363100    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.363172 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.363133    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-rvzp6\""
Jul 06 23:57:44.382647 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.382624    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ztxlq"]
Jul 06 23:57:44.382750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.382657    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"]
Jul 06 23:57:44.382750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.382672    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"]
Jul 06 23:57:44.382750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.382691    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5fb4c889c7-cwvmd"]
Jul 06 23:57:44.382750 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.382737    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.385327 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.385285    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.385442 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.385414    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.385520 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.385502    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 06 23:57:44.385582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.385523    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5ff6b\""
Jul 06 23:57:44.406786 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.406762    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fb4c889c7-cwvmd"]
Jul 06 23:57:44.406910 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.406856    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.409273 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409254    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 06 23:57:44.409386 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409366    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.409490 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409475    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rwfw6\""
Jul 06 23:57:44.409620 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409604    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 06 23:57:44.409699 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409666    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 06 23:57:44.409754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409691    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 06 23:57:44.409754 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.409722    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.419862 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.419842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.419953 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.419869    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.419953 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.419899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.420048 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.419957    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-snapshots\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.420048 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.419985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shdk5\" (UniqueName: \"kubernetes.io/projected/16821c74-4f6c-4568-bb61-8acbab454769-kube-api-access-shdk5\") pod \"volume-data-source-validator-5d456fdbcd-2xc2q\" (UID: \"16821c74-4f6c-4568-bb61-8acbab454769\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"
Jul 06 23:57:44.420048 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420025    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.420161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420050    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-tmp\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.420161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420070    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpq7v\" (UniqueName: \"kubernetes.io/projected/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-kube-api-access-cpq7v\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.420161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420131    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.420326 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420176    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/399f91a1-6840-42c2-b390-bb3e18779244-serving-cert\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.420326 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420228    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-default-certificate\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.420326 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420261    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8gzh\" (UniqueName: \"kubernetes.io/projected/78f1904f-f26e-4442-8ae8-3d1e5bce622d-kube-api-access-r8gzh\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.420326 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420288    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzmbc\" (UniqueName: \"kubernetes.io/projected/e82e9c05-b03b-461b-8959-3229ba134a61-kube-api-access-nzmbc\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.420502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420339    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlwhj\" (UniqueName: \"kubernetes.io/projected/399f91a1-6840-42c2-b390-bb3e18779244-kube-api-access-tlwhj\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.420502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420363    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-stats-auth\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.420502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.420502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420418    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.420502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.420433    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.420666 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.420514    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:44.420666 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.420561    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:44.920549647 +0000 UTC m=+50.245552909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:44.422545 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.422527    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-fbxck"]
Jul 06 23:57:44.434018 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.433995    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpq7v\" (UniqueName: \"kubernetes.io/projected/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-kube-api-access-cpq7v\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.441766 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.441703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.442898 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.442875    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-fbxck"]
Jul 06 23:57:44.445729 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.445709    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 06 23:57:44.445823 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.445732    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-67ftk\""
Jul 06 23:57:44.445823 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.445717    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 06 23:57:44.446686 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.446670    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:57:44.446763 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.446688    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 06 23:57:44.449645 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.449623    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 06 23:57:44.503350 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.503232    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:44.521278 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-tmp\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521395 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcskt\" (UniqueName: \"kubernetes.io/projected/05d4aaed-3119-474c-acc7-b981e197aeb0-kube-api-access-wcskt\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.521395 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521395 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/399f91a1-6840-42c2-b390-bb3e18779244-serving-cert\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521395 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-default-certificate\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.521395 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8gzh\" (UniqueName: \"kubernetes.io/projected/78f1904f-f26e-4442-8ae8-3d1e5bce622d-kube-api-access-r8gzh\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.521644 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzmbc\" (UniqueName: \"kubernetes.io/projected/e82e9c05-b03b-461b-8959-3229ba134a61-kube-api-access-nzmbc\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.521644 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-config\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.521644 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521542    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlwhj\" (UniqueName: \"kubernetes.io/projected/399f91a1-6840-42c2-b390-bb3e18779244-kube-api-access-tlwhj\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521644 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-stats-auth\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.521644 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521621    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521733    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-trusted-ca\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.521750    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-snapshots\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shdk5\" (UniqueName: \"kubernetes.io/projected/16821c74-4f6c-4568-bb61-8acbab454769-kube-api-access-shdk5\") pod \"volume-data-source-validator-5d456fdbcd-2xc2q\" (UID: \"16821c74-4f6c-4568-bb61-8acbab454769\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.521828    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:45.021807408 +0000 UTC m=+50.346810671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521750    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-tmp\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.521873    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:45.021856237 +0000 UTC m=+50.346859514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:44.521949 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d4aaed-3119-474c-acc7-b981e197aeb0-serving-cert\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.522446 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.521958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.522446 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.522031    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:44.522446 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.522071    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:45.022057931 +0000 UTC m=+50.347061207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:57:44.522606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.522493    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.522606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.522525    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/399f91a1-6840-42c2-b390-bb3e18779244-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.522606 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.522528    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/399f91a1-6840-42c2-b390-bb3e18779244-snapshots\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.523872 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.523848    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/399f91a1-6840-42c2-b390-bb3e18779244-serving-cert\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.523972 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.523918    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-stats-auth\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.524251 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.524233    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-default-certificate\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.530649 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.530610    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlwhj\" (UniqueName: \"kubernetes.io/projected/399f91a1-6840-42c2-b390-bb3e18779244-kube-api-access-tlwhj\") pod \"insights-operator-5c847dcfc4-ztxlq\" (UID: \"399f91a1-6840-42c2-b390-bb3e18779244\") " pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.530777 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.530686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzmbc\" (UniqueName: \"kubernetes.io/projected/e82e9c05-b03b-461b-8959-3229ba134a61-kube-api-access-nzmbc\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:44.530777 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.530724    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8gzh\" (UniqueName: \"kubernetes.io/projected/78f1904f-f26e-4442-8ae8-3d1e5bce622d-kube-api-access-r8gzh\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:44.530914 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.530875    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shdk5\" (UniqueName: \"kubernetes.io/projected/16821c74-4f6c-4568-bb61-8acbab454769-kube-api-access-shdk5\") pod \"volume-data-source-validator-5d456fdbcd-2xc2q\" (UID: \"16821c74-4f6c-4568-bb61-8acbab454769\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"
Jul 06 23:57:44.603675 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.603643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:44.622866 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.622834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-trusted-ca\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.623005 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.622910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d4aaed-3119-474c-acc7-b981e197aeb0-serving-cert\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.623005 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.622959    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcskt\" (UniqueName: \"kubernetes.io/projected/05d4aaed-3119-474c-acc7-b981e197aeb0-kube-api-access-wcskt\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.623129 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.623006    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-config\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.623665 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.623643    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-config\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.623820 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.623804    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/05d4aaed-3119-474c-acc7-b981e197aeb0-trusted-ca\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.625221 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.625203    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/05d4aaed-3119-474c-acc7-b981e197aeb0-serving-cert\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.630975 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.630951    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcskt\" (UniqueName: \"kubernetes.io/projected/05d4aaed-3119-474c-acc7-b981e197aeb0-kube-api-access-wcskt\") pod \"console-operator-68b757fb67-fbxck\" (UID: \"05d4aaed-3119-474c-acc7-b981e197aeb0\") " pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.654837 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.654817    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq"
Jul 06 23:57:44.669941 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.669920    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"
Jul 06 23:57:44.751421 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.751377    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:44.825664 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.825627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:57:44.825808 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.825684    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:57:44.825848 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.825828    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:57:44.825881 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.825837    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:57:44.825881 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.825867    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:57:44.825938 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.825895    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.825876967 +0000 UTC m=+66.150880246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:57:44.825938 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.825932    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.82590912 +0000 UTC m=+66.150912436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:57:44.830268 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.830243    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-ztxlq"]
Jul 06 23:57:44.832032 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:57:44.831994    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399f91a1_6840_42c2_b390_bb3e18779244.slice/crio-5f9ac64ea0fdf418645e62d0d68fe089cdb51dba28a51d5cdd1debc1767559ee WatchSource:0}: Error finding container 5f9ac64ea0fdf418645e62d0d68fe089cdb51dba28a51d5cdd1debc1767559ee: Status 404 returned error can't find the container with id 5f9ac64ea0fdf418645e62d0d68fe089cdb51dba28a51d5cdd1debc1767559ee
Jul 06 23:57:44.854870 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.854842    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q"]
Jul 06 23:57:44.856510 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:57:44.856485    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16821c74_4f6c_4568_bb61_8acbab454769.slice/crio-c6b40034fb17ae9dd02628cd457b4f00aa221b35ff9e007c126710d8a76e5947 WatchSource:0}: Error finding container c6b40034fb17ae9dd02628cd457b4f00aa221b35ff9e007c126710d8a76e5947: Status 404 returned error can't find the container with id c6b40034fb17ae9dd02628cd457b4f00aa221b35ff9e007c126710d8a76e5947
Jul 06 23:57:44.916721 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.916692    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-fbxck"]
Jul 06 23:57:44.919877 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:57:44.919843    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d4aaed_3119_474c_acc7_b981e197aeb0.slice/crio-b86e6dfe5d4fb3fec9cb4215e48d810a42787fa898d558fed976540b93324421 WatchSource:0}: Error finding container b86e6dfe5d4fb3fec9cb4215e48d810a42787fa898d558fed976540b93324421: Status 404 returned error can't find the container with id b86e6dfe5d4fb3fec9cb4215e48d810a42787fa898d558fed976540b93324421
Jul 06 23:57:44.926502 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:44.926484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:44.926638 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.926621    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:44.926688 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:44.926679    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:45.926665068 +0000 UTC m=+51.251668331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:45.027166 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.027079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:45.027166 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.027125    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.027185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.027259    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.027288    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.027336    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:46.027319953 +0000 UTC m=+51.352323218 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.027357    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:46.027351054 +0000 UTC m=+51.352354320 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:57:45.027424 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.027367    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:46.027361778 +0000 UTC m=+51.352365040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:57:45.162202 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.162173    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"]
Jul 06 23:57:45.174751 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.174722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.175274 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.175252    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"]
Jul 06 23:57:45.177324 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.177304    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 06 23:57:45.177422 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.177322    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 06 23:57:45.177422 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.177286    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 06 23:57:45.177422 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.177361    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-dw5s8\""
Jul 06 23:57:45.177422 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.177307    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 06 23:57:45.229054 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.229024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/699771fc-b797-47a3-9c37-81b0ebbfb0dd-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.229536 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.229074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/699771fc-b797-47a3-9c37-81b0ebbfb0dd-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.229536 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.229219    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4px7\" (UniqueName: \"kubernetes.io/projected/699771fc-b797-47a3-9c37-81b0ebbfb0dd-kube-api-access-l4px7\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.329771 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.329679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/699771fc-b797-47a3-9c37-81b0ebbfb0dd-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.329771 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.329762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/699771fc-b797-47a3-9c37-81b0ebbfb0dd-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.330148 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.329864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4px7\" (UniqueName: \"kubernetes.io/projected/699771fc-b797-47a3-9c37-81b0ebbfb0dd-kube-api-access-l4px7\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.330423 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.330394    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/699771fc-b797-47a3-9c37-81b0ebbfb0dd-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.330559 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.330530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:45.331975 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.331949    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/699771fc-b797-47a3-9c37-81b0ebbfb0dd-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.337371 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.337354    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4px7\" (UniqueName: \"kubernetes.io/projected/699771fc-b797-47a3-9c37-81b0ebbfb0dd-kube-api-access-l4px7\") pod \"kube-storage-version-migrator-operator-846bf668f7-4ckw6\" (UID: \"699771fc-b797-47a3-9c37-81b0ebbfb0dd\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.353996 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.353966    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" event={"ID":"05d4aaed-3119-474c-acc7-b981e197aeb0","Type":"ContainerStarted","Data":"b86e6dfe5d4fb3fec9cb4215e48d810a42787fa898d558fed976540b93324421"}
Jul 06 23:57:45.354921 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.354901    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q" event={"ID":"16821c74-4f6c-4568-bb61-8acbab454769","Type":"ContainerStarted","Data":"c6b40034fb17ae9dd02628cd457b4f00aa221b35ff9e007c126710d8a76e5947"}
Jul 06 23:57:45.355913 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.355892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq" event={"ID":"399f91a1-6840-42c2-b390-bb3e18779244","Type":"ContainerStarted","Data":"5f9ac64ea0fdf418645e62d0d68fe089cdb51dba28a51d5cdd1debc1767559ee"}
Jul 06 23:57:45.485362 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.485195    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"
Jul 06 23:57:45.650871 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.650809    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6"]
Jul 06 23:57:45.654675 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:57:45.654643    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod699771fc_b797_47a3_9c37_81b0ebbfb0dd.slice/crio-66b74d0b71cfa995b70ec214246bb03bb8107e3cefbfbf53538592403f8fa63c WatchSource:0}: Error finding container 66b74d0b71cfa995b70ec214246bb03bb8107e3cefbfbf53538592403f8fa63c: Status 404 returned error can't find the container with id 66b74d0b71cfa995b70ec214246bb03bb8107e3cefbfbf53538592403f8fa63c
Jul 06 23:57:45.936001 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:45.935962    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:45.936199 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.936148    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:45.936255 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:45.936229    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:47.936207672 +0000 UTC m=+53.261210940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:46.037582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:46.037539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:46.037768 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:46.037601    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:46.037768 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:46.037710    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:57:46.037768 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:46.037746    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.037725372 +0000 UTC m=+53.362728649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:46.037980 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:46.037773    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.037762427 +0000 UTC m=+53.362765720 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:57:46.037980 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:46.037800    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:46.037980 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:46.037713    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:46.037980 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:46.037845    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:48.037830036 +0000 UTC m=+53.362833316 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:57:46.359812 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:46.359728    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6" event={"ID":"699771fc-b797-47a3-9c37-81b0ebbfb0dd","Type":"ContainerStarted","Data":"66b74d0b71cfa995b70ec214246bb03bb8107e3cefbfbf53538592403f8fa63c"}
Jul 06 23:57:47.954539 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:47.954501    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:47.954943 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:47.954616    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:47.954943 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:47.954676    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.954659742 +0000 UTC m=+57.279663024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:48.056054 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:48.056016    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:48.056197 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:48.056073    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:48.056197 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:48.056145    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:48.056326 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:48.056200    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:57:48.056326 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:48.056280    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.056257364 +0000 UTC m=+57.381260629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:48.056326 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:48.056317    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:48.056482 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:48.056325    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.056314337 +0000 UTC m=+57.381317601 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:57:48.056482 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:48.056385    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.056367448 +0000 UTC m=+57.381370714 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:57:48.081973 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:48.081944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:48.173046 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:48.172996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.369577 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.369551    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/0.log"
Jul 06 23:57:49.369996 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.369591    2583 generic.go:356] "Generic (PLEG): container finished" podID="05d4aaed-3119-474c-acc7-b981e197aeb0" containerID="57f05900bb8777454f3f6a6011133656d97d8c2f068f2a7cdfb460cc90922894" exitCode=255
Jul 06 23:57:49.369996 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.369664    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" event={"ID":"05d4aaed-3119-474c-acc7-b981e197aeb0","Type":"ContainerDied","Data":"57f05900bb8777454f3f6a6011133656d97d8c2f068f2a7cdfb460cc90922894"}
Jul 06 23:57:49.369996 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.369921    2583 scope.go:117] "RemoveContainer" containerID="57f05900bb8777454f3f6a6011133656d97d8c2f068f2a7cdfb460cc90922894"
Jul 06 23:57:49.371274 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.371247    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q" event={"ID":"16821c74-4f6c-4568-bb61-8acbab454769","Type":"ContainerStarted","Data":"c95226c8b521b0b68f231d12732fb893eadc186a91a00feb589e9cbc5eb51fee"}
Jul 06 23:57:49.377925 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.377897    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq" event={"ID":"399f91a1-6840-42c2-b390-bb3e18779244","Type":"ContainerStarted","Data":"4bd4aa766a28c8bdbededb28017c930c27490cc3bd8918bd3a76fe6fbb2cc4e7"}
Jul 06 23:57:49.379410 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.379383    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6" event={"ID":"699771fc-b797-47a3-9c37-81b0ebbfb0dd","Type":"ContainerStarted","Data":"371849832b6ed9cc1a52279dc25af310bd36c92932e97425493798dc652af791"}
Jul 06 23:57:49.405552 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.405497    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq" podStartSLOduration=1.79684702 podStartE2EDuration="5.405477477s" totalImagesPullingTime="3.608630457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:49.40398234 +0000 UTC m=+54.728985624" watchObservedRunningTime="2026-07-06 23:57:49.405477477 +0000 UTC m=+54.730480764"
Jul 06 23:57:49.419980 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.419932    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-2xc2q" podStartSLOduration=1.836367932 podStartE2EDuration="5.419916663s" totalImagesPullingTime="3.583548731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:49.419346166 +0000 UTC m=+54.744349451" watchObservedRunningTime="2026-07-06 23:57:49.419916663 +0000 UTC m=+54.744919995"
Jul 06 23:57:49.437715 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.437665    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6" podStartSLOduration=1.616054023 podStartE2EDuration="4.437647974s" totalImagesPullingTime="2.821593951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:57:49.436971477 +0000 UTC m=+54.761974761" watchObservedRunningTime="2026-07-06 23:57:49.437647974 +0000 UTC m=+54.762651259"
Jul 06 23:57:49.500085 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.500051    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:49.558798 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:49.558760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.384785 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.384752    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 06 23:57:50.385218 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.385202    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/0.log"
Jul 06 23:57:50.385264 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.385242    2583 generic.go:356] "Generic (PLEG): container finished" podID="05d4aaed-3119-474c-acc7-b981e197aeb0" containerID="39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7" exitCode=255
Jul 06 23:57:50.385358 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.385337    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" event={"ID":"05d4aaed-3119-474c-acc7-b981e197aeb0","Type":"ContainerDied","Data":"39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7"}
Jul 06 23:57:50.385436 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.385392    2583 scope.go:117] "RemoveContainer" containerID="57f05900bb8777454f3f6a6011133656d97d8c2f068f2a7cdfb460cc90922894"
Jul 06 23:57:50.385637 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.385619    2583 scope.go:117] "RemoveContainer" containerID="39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7"
Jul 06 23:57:50.385887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:50.385815    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-fbxck_openshift-console-operator(05d4aaed-3119-474c-acc7-b981e197aeb0)\"" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" podUID="05d4aaed-3119-474c-acc7-b981e197aeb0"
Jul 06 23:57:50.523973 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.523925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.625376 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.625344    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.735695 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.735605    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zgr8w"]
Jul 06 23:57:50.738691 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.738662    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.741040 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.741017    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 23:57:50.741176 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.741152    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 23:57:50.741276 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.741237    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lqq6\""
Jul 06 23:57:50.753449 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.753427    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgr8w"]
Jul 06 23:57:50.780002 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.779974    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-crio-socket\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.780002 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.780004    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-data-volume\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.780194 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.780022    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.780194 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.780065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4spdt\" (UniqueName: \"kubernetes.io/projected/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-api-access-4spdt\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.780194 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.780159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.797581 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.797551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:50.881312 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881262    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4spdt\" (UniqueName: \"kubernetes.io/projected/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-api-access-4spdt\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881510 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881583 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-crio-socket\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881583 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-data-volume\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881583 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881554    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881703 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:50.881587    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:50.881703 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881643    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-crio-socket\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.881703 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:50.881649    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:57:50.881703 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:50.881662    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.381643639 +0000 UTC m=+56.706646920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:50.881834 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:50.881711    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:51.381699922 +0000 UTC m=+56.706703187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : configmap "kube-rbac-proxy" not found
Jul 06 23:57:50.881906 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.881888    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-data-volume\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.894091 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.894063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4spdt\" (UniqueName: \"kubernetes.io/projected/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-api-access-4spdt\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:50.912582 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:50.912530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.096877 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.096773    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.199131 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.199094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.294414 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.294385    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.386595 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.386560    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:51.387102 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.386688    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:51.387102 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.386702    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:51.387102 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.386762    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.386746452 +0000 UTC m=+57.711749734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:51.387102 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.386784    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:57:51.387102 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.386836    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:52.386820464 +0000 UTC m=+57.711823740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : configmap "kube-rbac-proxy" not found
Jul 06 23:57:51.389758 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.389740    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 06 23:57:51.390074 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.390056    2583 scope.go:117] "RemoveContainer" containerID="39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7"
Jul 06 23:57:51.390238 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.390219    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-fbxck_openshift-console-operator(05d4aaed-3119-474c-acc7-b981e197aeb0)\"" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" podUID="05d4aaed-3119-474c-acc7-b981e197aeb0"
Jul 06 23:57:51.562922 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.562889    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.664138 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.664047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:51.993868 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:51.993775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:57:51.994021 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.993934    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:51.994021 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:51.994000    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:57:59.993984877 +0000 UTC m=+65.318988141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:57:52.094637 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.094582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:52.094810 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.094686    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:57:52.094810 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.094753    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.094735943 +0000 UTC m=+65.419739211 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:57:52.094810 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.094792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:57:52.094810 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.094805    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:57:52.094971 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.094844    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.094832881 +0000 UTC m=+65.419836146 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:57:52.094971 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.094864    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:57:52.094971 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.094891    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:58:00.094883113 +0000 UTC m=+65.419886375 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:57:52.127523 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.127495    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-crvxn_579e3da3-6a24-4a7d-a719-519db369ad56/dns-node-resolver/0.log"
Jul 06 23:57:52.397440 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.397403    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:52.397887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.397521    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:57:52.397887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.397578    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:54.397564048 +0000 UTC m=+59.722567314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : configmap "kube-rbac-proxy" not found
Jul 06 23:57:52.397887 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:52.397524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:52.397887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.397620    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:52.397887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:52.397674    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:54.3976584 +0000 UTC m=+59.722661676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:53.527603 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:53.527571    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c5v2z_d32f0935-e944-44ed-a4c6-2f0e8c58376d/node-ca/0.log"
Jul 06 23:57:54.127189 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.127158    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ccxln_e2f224ba-1efd-4b24-b548-44b7c311c4f3/serve-healthcheck-canary/0.log"
Jul 06 23:57:54.250347 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.250316    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:54.351043 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.350997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:54.414308 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.414214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:54.414456 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.414319    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:54.414456 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:54.414379    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:54.414456 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:54.414419    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:57:54.414456 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:54.414442    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:58.414427463 +0000 UTC m=+63.739430724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:54.414589 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:54.414461    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:57:58.41444941 +0000 UTC m=+63.739452672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : configmap "kube-rbac-proxy" not found
Jul 06 23:57:54.451554 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.451521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:54.752367 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.752267    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:54.752367 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.752364    2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:57:54.752864 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.752805    2583 scope.go:117] "RemoveContainer" containerID="39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7"
Jul 06 23:57:54.753009 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:54.752987    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-fbxck_openshift-console-operator(05d4aaed-3119-474c-acc7-b981e197aeb0)\"" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" podUID="05d4aaed-3119-474c-acc7-b981e197aeb0"
Jul 06 23:57:54.912426 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:54.912391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:55.013025 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:55.012938    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:55.112914 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:55.112877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:55.306981 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:55.306909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x56f2"
Jul 06 23:57:55.462753 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:55.462723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:55.529728 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:55.529694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:56.642030 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:56.641995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:56.695731 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:56.695699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:57.737794 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:57.737754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:57:58.452165 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:58.452124    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:57:58.452401 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:58.452331    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:58.452401 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:58.452386    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 23:57:58.452522 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:58.452404    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:06.452385293 +0000 UTC m=+71.777388569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:57:58.452522 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:57:58.452433    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:06.452421574 +0000 UTC m=+71.777424836 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : configmap "kube-rbac-proxy" not found
Jul 06 23:57:58.452522 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:57:58.452334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:00.066595 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.066546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:58:00.067001 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.066692    2583 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 06 23:58:00.067001 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.066758    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls podName:a1cc70ee-c768-4fbc-b89e-d6e977fe751c nodeName:}" failed. No retries permitted until 2026-07-06 23:58:16.0667435 +0000 UTC m=+81.391746762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-nc99n" (UID: "a1cc70ee-c768-4fbc-b89e-d6e977fe751c") : secret "cluster-monitoring-operator-tls" not found
Jul 06 23:58:00.167326 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.167267    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:00.167496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.167378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:58:00.167496 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.167409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:00.167496 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.167412    2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 06 23:58:00.167496 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.167475    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:16.167460352 +0000 UTC m=+81.492463632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : secret "router-metrics-certs-default" not found
Jul 06 23:58:00.167633 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.167509    2583 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 06 23:58:00.167633 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.167521    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle podName:e82e9c05-b03b-461b-8959-3229ba134a61 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:16.167510045 +0000 UTC m=+81.492513306 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle") pod "router-default-5fb4c889c7-cwvmd" (UID: "e82e9c05-b03b-461b-8959-3229ba134a61") : configmap references non-existent config key: service-ca.crt
Jul 06 23:58:00.167633 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.167564    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls podName:78f1904f-f26e-4442-8ae8-3d1e5bce622d nodeName:}" failed. No retries permitted until 2026-07-06 23:58:16.167553281 +0000 UTC m=+81.492556543 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-5s582" (UID: "78f1904f-f26e-4442-8ae8-3d1e5bce622d") : secret "samples-operator-tls" not found
Jul 06 23:58:00.459302 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.459267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:00.872997 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.872966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:00.873140 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.873007    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:00.873140 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.873123    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 23:58:00.873208 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.873140    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 23:58:00.873208 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.873143    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-f9b979749-tkpfg: secret "image-registry-tls" not found
Jul 06 23:58:00.873208 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.873190    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls podName:9745e43d-f62d-476a-980f-54ad7ccf8c8f nodeName:}" failed. No retries permitted until 2026-07-06 23:58:32.873176812 +0000 UTC m=+98.198180074 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls") pod "dns-default-qrhjm" (UID: "9745e43d-f62d-476a-980f-54ad7ccf8c8f") : secret "dns-default-metrics-tls" not found
Jul 06 23:58:00.873208 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.873204    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls podName:cce091bc-867c-422e-8a3a-0af53ae45b93 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:32.873197881 +0000 UTC m=+98.198201142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls") pod "image-registry-f9b979749-tkpfg" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93") : secret "image-registry-tls" not found
Jul 06 23:58:00.974283 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.974247    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:58:00.974453 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.974360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:58:00.977008 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.976991    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 23:58:00.977060 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.976995    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 23:58:00.985410 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.985391    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 23:58:00.985473 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:00.985462    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs podName:91a84a2f-d2eb-419f-b778-bab4fe49a581 nodeName:}" failed. No retries permitted until 2026-07-06 23:59:04.985442167 +0000 UTC m=+130.310445429 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs") pod "network-metrics-daemon-j6rtl" (UID: "91a84a2f-d2eb-419f-b778-bab4fe49a581") : secret "metrics-daemon-secret" not found
Jul 06 23:58:00.987536 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:00.987509    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/534cd6f5-0f18-4837-90e0-e83f0a16cefe-original-pull-secret\") pod \"global-pull-secret-syncer-hxjb2\" (UID: \"534cd6f5-0f18-4837-90e0-e83f0a16cefe\") " pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:58:01.075054 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.075010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:58:01.078128 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.078111    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 23:58:01.088093 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.088076    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 23:58:01.095824 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.095791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.098057 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.098040    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjkc6\" (UniqueName: \"kubernetes.io/projected/e7da8523-bde6-4880-8edd-5bc6f5001d4d-kube-api-access-vjkc6\") pod \"network-check-target-fxpmb\" (UID: \"e7da8523-bde6-4880-8edd-5bc6f5001d4d\") " pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:58:01.194699 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.194621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.267658 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.267628    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v6xkb\""
Jul 06 23:58:01.275065 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.275042    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:58:01.277375 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.277359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hxjb2"
Jul 06 23:58:01.305281 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.305244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.403370 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.403335    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.450759 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.450685    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fxpmb"]
Jul 06 23:58:01.454547 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:01.454512    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7da8523_bde6_4880_8edd_5bc6f5001d4d.slice/crio-fb1bbb4996d1282b0bf6bd92215d0593856f5a3a5ea319bd0f562079c8ce3d9a WatchSource:0}: Error finding container fb1bbb4996d1282b0bf6bd92215d0593856f5a3a5ea319bd0f562079c8ce3d9a: Status 404 returned error can't find the container with id fb1bbb4996d1282b0bf6bd92215d0593856f5a3a5ea319bd0f562079c8ce3d9a
Jul 06 23:58:01.488340 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.488308    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hxjb2"]
Jul 06 23:58:01.491171 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:01.491142    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534cd6f5_0f18_4837_90e0_e83f0a16cefe.slice/crio-d0bf196228015237a4f3474e9b80cfbf9014d9bdc0a79a11929eead47a96304c WatchSource:0}: Error finding container d0bf196228015237a4f3474e9b80cfbf9014d9bdc0a79a11929eead47a96304c: Status 404 returned error can't find the container with id d0bf196228015237a4f3474e9b80cfbf9014d9bdc0a79a11929eead47a96304c
Jul 06 23:58:01.493117 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.493092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.576707 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.576665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:01.789154 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:01.789048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:02.420801 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:02.420762    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hxjb2" event={"ID":"534cd6f5-0f18-4837-90e0-e83f0a16cefe","Type":"ContainerStarted","Data":"d0bf196228015237a4f3474e9b80cfbf9014d9bdc0a79a11929eead47a96304c"}
Jul 06 23:58:02.421983 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:02.421944    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxpmb" event={"ID":"e7da8523-bde6-4880-8edd-5bc6f5001d4d","Type":"ContainerStarted","Data":"fb1bbb4996d1282b0bf6bd92215d0593856f5a3a5ea319bd0f562079c8ce3d9a"}
Jul 06 23:58:02.589018 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:02.588982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:02.988428 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:02.988390    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:03.191763 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:03.191727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:03.685040 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:03.685003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:03.785315 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:03.785268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:04.183634 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:04.183595    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:04.376466 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:04.376430    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:04.430609 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:04.430553    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:04.776411 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:04.776374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:05.154078 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:05.154044    2583 scope.go:117] "RemoveContainer" containerID="39568dd760b31cb2f06107017317f2510b8ed53a8c72ba42b311a3a89e9b01c7"
Jul 06 23:58:05.187006 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:05.186956    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:05.602056 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:05.601967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:05.981874 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:05.981843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.176694 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.176653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.229545 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.229512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.437487 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.437450    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fxpmb" event={"ID":"e7da8523-bde6-4880-8edd-5bc6f5001d4d","Type":"ContainerStarted","Data":"e144548172ae387e05297b597e17d4ded0ae86f9311f73cc4b9bfc55fe7eeec6"}
Jul 06 23:58:06.437672 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.437638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:58:06.439448 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.439423    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 06 23:58:06.439559 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.439526    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" event={"ID":"05d4aaed-3119-474c-acc7-b981e197aeb0","Type":"ContainerStarted","Data":"52fc300d1f6bafb9e59f84c5eec627acf7447d01489162fad034431fa30623cd"}
Jul 06 23:58:06.439856 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.439828    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:58:06.441379 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.441350    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hxjb2" event={"ID":"534cd6f5-0f18-4837-90e0-e83f0a16cefe","Type":"ContainerStarted","Data":"51789d6d5654f1829d451d56a6bb79248febe1bb1d05bd914ee0674c424cc291"}
Jul 06 23:58:06.446665 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.446636    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-fbxck"
Jul 06 23:58:06.467308 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.464568    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fxpmb" podStartSLOduration=66.898225636 podStartE2EDuration="1m11.464548722s" totalImagesPullingTime="4.566323086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:06.463271348 +0000 UTC m=+71.788274624" watchObservedRunningTime="2026-07-06 23:58:06.464548722 +0000 UTC m=+71.789552007"
Jul 06 23:58:06.480646 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.480514    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hxjb2" podStartSLOduration=66.943247866 podStartE2EDuration="1m11.480494177s" totalImagesPullingTime="4.537246311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:06.480490251 +0000 UTC m=+71.805493536" watchObservedRunningTime="2026-07-06 23:58:06.480494177 +0000 UTC m=+71.805497462"
Jul 06 23:58:06.500222 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.500091    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-fbxck" podStartSLOduration=18.978207623 podStartE2EDuration="22.500071595s" totalImagesPullingTime="3.521863972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:06.499966906 +0000 UTC m=+71.824970191" watchObservedRunningTime="2026-07-06 23:58:06.500071595 +0000 UTC m=+71.825074878"
Jul 06 23:58:06.523564 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.523530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:06.523736 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.523680    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:06.523825 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:06.523805    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 23:58:06.523887 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:06.523868    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls podName:9918fb3f-6f60-4983-94fb-7c5ef286f0d0 nodeName:}" failed. No retries permitted until 2026-07-06 23:58:22.523851765 +0000 UTC m=+87.848855038 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zgr8w" (UID: "9918fb3f-6f60-4983-94fb-7c5ef286f0d0") : secret "insights-runtime-extractor-tls" not found
Jul 06 23:58:06.524135 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.524108    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:06.592263 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.592217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.692550 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.692465    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.793696 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.793664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.897161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.897118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:06.995446 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:06.995356    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.096971 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.096932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.271558 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.271477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.327766 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.327734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.412831 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.412801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.676563 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.676534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.733170 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.733133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:07.801911 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:07.801872    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:08.179232 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:08.179198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:08.281053 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:08.281012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:08.384495 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:08.384455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:08.498585 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:08.498499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:08.983623 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:08.983591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.186547 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.186501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.250433 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.250351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.314972 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.314943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.415303 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.415263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.577969 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.577876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:09.779639 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:09.779596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:10.191276 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:10.191241    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:10.387733 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:10.387697    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:10.981107 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:10.981077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:11.582197 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:11.582156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:11.778871 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:11.778829    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:11.978104 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:11.978070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:12.177743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:12.177705    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:14.907306 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:14.907267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:14.961573 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:14.961533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:15.796933 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:15.796897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:15.961267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:15.961229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.102277 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.102191    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:58:16.104601 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.104575    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a1cc70ee-c768-4fbc-b89e-d6e977fe751c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-nc99n\" (UID: \"a1cc70ee-c768-4fbc-b89e-d6e977fe751c\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:58:16.203063 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.203030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:16.203251 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.203084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:58:16.203251 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.203109    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:16.203807 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.203787    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e82e9c05-b03b-461b-8959-3229ba134a61-service-ca-bundle\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:16.205428 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.205402    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e82e9c05-b03b-461b-8959-3229ba134a61-metrics-certs\") pod \"router-default-5fb4c889c7-cwvmd\" (UID: \"e82e9c05-b03b-461b-8959-3229ba134a61\") " pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:16.205549 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.205527    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/78f1904f-f26e-4442-8ae8-3d1e5bce622d-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-5s582\" (UID: \"78f1904f-f26e-4442-8ae8-3d1e5bce622d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:58:16.217906 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.217888    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rwfw6\""
Jul 06 23:58:16.225457 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.225439    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:16.356769 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.356697    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5fb4c889c7-cwvmd"]
Jul 06 23:58:16.360035 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:16.360006    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82e9c05_b03b_461b_8959_3229ba134a61.slice/crio-bb8b04aca8b62f66d3533198f1e3d84e6c2674d39e756e85f7be23ae86c199a9 WatchSource:0}: Error finding container bb8b04aca8b62f66d3533198f1e3d84e6c2674d39e756e85f7be23ae86c199a9: Status 404 returned error can't find the container with id bb8b04aca8b62f66d3533198f1e3d84e6c2674d39e756e85f7be23ae86c199a9
Jul 06 23:58:16.377706 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.377686    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rt8s4\""
Jul 06 23:58:16.385116 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.385093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"
Jul 06 23:58:16.475575 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.475541    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd" event={"ID":"e82e9c05-b03b-461b-8959-3229ba134a61","Type":"ContainerStarted","Data":"42be6972012557e6e08d0fffc07f6c8ea5b69c8d1e60a650ef142f35d488c660"}
Jul 06 23:58:16.475712 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.475603    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd" event={"ID":"e82e9c05-b03b-461b-8959-3229ba134a61","Type":"ContainerStarted","Data":"bb8b04aca8b62f66d3533198f1e3d84e6c2674d39e756e85f7be23ae86c199a9"}
Jul 06 23:58:16.486217 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.486185    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.493815 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.493785    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5ff6b\""
Jul 06 23:58:16.499684 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.499641    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd" podStartSLOduration=32.499623721 podStartE2EDuration="32.499623721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:16.499246541 +0000 UTC m=+81.824249837" watchObservedRunningTime="2026-07-06 23:58:16.499623721 +0000 UTC m=+81.824627004"
Jul 06 23:58:16.501282 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.501248    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"
Jul 06 23:58:16.522442 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.522361    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n"]
Jul 06 23:58:16.524681 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:16.524653    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cc70ee_c768_4fbc_b89e_d6e977fe751c.slice/crio-7acb0f2d1c1fc99d77f5c29ac063212d2ed5b61ef98fdbabce95d16cd54703e3 WatchSource:0}: Error finding container 7acb0f2d1c1fc99d77f5c29ac063212d2ed5b61ef98fdbabce95d16cd54703e3: Status 404 returned error can't find the container with id 7acb0f2d1c1fc99d77f5c29ac063212d2ed5b61ef98fdbabce95d16cd54703e3
Jul 06 23:58:16.584374 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.584343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.632697 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.632650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582"]
Jul 06 23:58:16.863518 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.863393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:16.961554 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:16.961461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:17.226095 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.225966    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:17.229048 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.229002    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:17.481389 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.481277    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582" event={"ID":"78f1904f-f26e-4442-8ae8-3d1e5bce622d","Type":"ContainerStarted","Data":"fff0320b3ff7a47a830ad191dc4b5a8b62e3331473da7be8fc0679ce128b68fd"}
Jul 06 23:58:17.483257 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.483210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n" event={"ID":"a1cc70ee-c768-4fbc-b89e-d6e977fe751c","Type":"ContainerStarted","Data":"7acb0f2d1c1fc99d77f5c29ac063212d2ed5b61ef98fdbabce95d16cd54703e3"}
Jul 06 23:58:17.483463 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.483442    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:17.484792 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:17.484773    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5fb4c889c7-cwvmd"
Jul 06 23:58:18.089468 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:18.089415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:18.265892 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:18.265859    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:18.328119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:18.328082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:18.978851 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:18.978748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.076624 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.076594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.184611 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.184580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.282275 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.282191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.496639 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.496602    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582" event={"ID":"78f1904f-f26e-4442-8ae8-3d1e5bce622d","Type":"ContainerStarted","Data":"f68f714a948cd3046d2e89ad8f7f7cdddf2427a9286d48e1c16e67f218ad387b"}
Jul 06 23:58:19.496814 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.496688    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582" event={"ID":"78f1904f-f26e-4442-8ae8-3d1e5bce622d","Type":"ContainerStarted","Data":"bdcc8c54628b1ad0ea48be18210edddd998d40d05c0c97dc40ef6354d6f2f7c9"}
Jul 06 23:58:19.497985 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.497958    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n" event={"ID":"a1cc70ee-c768-4fbc-b89e-d6e977fe751c","Type":"ContainerStarted","Data":"b097c520464c320b31620c7c44842b8e809d3bd9f2c3e385c160cc22b8ad2711"}
Jul 06 23:58:19.512857 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.512810    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-5s582" podStartSLOduration=33.54282903 podStartE2EDuration="35.512792639s" totalImagesPullingTime="1.969963609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:19.512572425 +0000 UTC m=+84.837575709" watchObservedRunningTime="2026-07-06 23:58:19.512792639 +0000 UTC m=+84.837795924"
Jul 06 23:58:19.529534 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.529485    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-nc99n" podStartSLOduration=33.421751486 podStartE2EDuration="35.529472035s" totalImagesPullingTime="2.107720549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:19.528825182 +0000 UTC m=+84.853828467" watchObservedRunningTime="2026-07-06 23:58:19.529472035 +0000 UTC m=+84.854475322"
Jul 06 23:58:19.563236 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.563154    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.663325 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.663270    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:19.714770 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:19.714735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:21.995735 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:21.995700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:22.065690 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.065654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:22.557559 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.557528    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:22.559772 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.559746    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9918fb3f-6f60-4983-94fb-7c5ef286f0d0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zgr8w\" (UID: \"9918fb3f-6f60-4983-94fb-7c5ef286f0d0\") " pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:22.851157 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.851067    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lqq6\""
Jul 06 23:58:22.859437 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.859413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zgr8w"
Jul 06 23:58:22.987689 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:22.987657    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zgr8w"]
Jul 06 23:58:22.990750 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:22.990720    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9918fb3f_6f60_4983_94fb_7c5ef286f0d0.slice/crio-e6071470a7f70152c4328c6159b734069013aaec64f0691820adcfa81d45ecd6 WatchSource:0}: Error finding container e6071470a7f70152c4328c6159b734069013aaec64f0691820adcfa81d45ecd6: Status 404 returned error can't find the container with id e6071470a7f70152c4328c6159b734069013aaec64f0691820adcfa81d45ecd6
Jul 06 23:58:23.515847 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:23.515760    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgr8w" event={"ID":"9918fb3f-6f60-4983-94fb-7c5ef286f0d0","Type":"ContainerStarted","Data":"e6071470a7f70152c4328c6159b734069013aaec64f0691820adcfa81d45ecd6"}
Jul 06 23:58:24.306498 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:24.306454    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:24.408148 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:24.408119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:25.186505 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:25.186464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:25.260316 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:25.260260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:25.954004 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:25.953959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:26.108532 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:26.108503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:26.169098 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:26.169012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:27.305039 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:27.305006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:27.403828 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:27.403793    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:27.532011 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:27.531928    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgr8w" event={"ID":"9918fb3f-6f60-4983-94fb-7c5ef286f0d0","Type":"ContainerStarted","Data":"0c969342c979b95ab8f1a4d9878d64c3cc53061bac0f299aa3f419339809f902"}
Jul 06 23:58:27.532011 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:27.531983    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgr8w" event={"ID":"9918fb3f-6f60-4983-94fb-7c5ef286f0d0","Type":"ContainerStarted","Data":"fb97df007b9e6fc99a616f782cc4ecaac64b233fc7f43a5713b745a24c623044"}
Jul 06 23:58:28.979732 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:28.979695    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:29.033323 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:29.033263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:30.502264 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:30.502228    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:30.546038 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:30.546002    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zgr8w" event={"ID":"9918fb3f-6f60-4983-94fb-7c5ef286f0d0","Type":"ContainerStarted","Data":"fb5a47bc9d761311eac70dc87a57a453d9b69e490254d76cf6d8fc2d07081682"}
Jul 06 23:58:30.568894 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:30.568844    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zgr8w" podStartSLOduration=33.866911776 podStartE2EDuration="40.568828212s" totalImagesPullingTime="6.701916436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:57:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:30.566949991 +0000 UTC m=+95.891953268" watchObservedRunningTime="2026-07-06 23:58:30.568828212 +0000 UTC m=+95.893831496"
Jul 06 23:58:30.571623 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:30.571593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:32.946275 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:32.946226    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:32.946275 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:32.946280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:32.948724 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:32.948694    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9745e43d-f62d-476a-980f-54ad7ccf8c8f-metrics-tls\") pod \"dns-default-qrhjm\" (UID: \"9745e43d-f62d-476a-980f-54ad7ccf8c8f\") " pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:32.948834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:32.948759    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"image-registry-f9b979749-tkpfg\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") " pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:33.201548 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.201470    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qtrrj\""
Jul 06 23:58:33.209649 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.209624    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:33.216228 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.216208    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-s7rq7\""
Jul 06 23:58:33.224377 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.224349    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:33.377589 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.377555    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:58:33.380586 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:33.380558    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce091bc_867c_422e_8a3a_0af53ae45b93.slice/crio-d8791481d6d16cbb7682298ed3e1ea0b3f810cc270ff26fdf9c58395df3b4a2f WatchSource:0}: Error finding container d8791481d6d16cbb7682298ed3e1ea0b3f810cc270ff26fdf9c58395df3b4a2f: Status 404 returned error can't find the container with id d8791481d6d16cbb7682298ed3e1ea0b3f810cc270ff26fdf9c58395df3b4a2f
Jul 06 23:58:33.391410 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.391385    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qrhjm"]
Jul 06 23:58:33.394338 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:33.394313    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9745e43d_f62d_476a_980f_54ad7ccf8c8f.slice/crio-6b200617a4e1aee9ec1dffbdffb5451adc3b131a4c66825ed3af5e336239c299 WatchSource:0}: Error finding container 6b200617a4e1aee9ec1dffbdffb5451adc3b131a4c66825ed3af5e336239c299: Status 404 returned error can't find the container with id 6b200617a4e1aee9ec1dffbdffb5451adc3b131a4c66825ed3af5e336239c299
Jul 06 23:58:33.500345 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.500244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:33.556339 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.556284    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrhjm" event={"ID":"9745e43d-f62d-476a-980f-54ad7ccf8c8f","Type":"ContainerStarted","Data":"6b200617a4e1aee9ec1dffbdffb5451adc3b131a4c66825ed3af5e336239c299"}
Jul 06 23:58:33.557830 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.557804    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" event={"ID":"cce091bc-867c-422e-8a3a-0af53ae45b93","Type":"ContainerStarted","Data":"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"}
Jul 06 23:58:33.557920 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.557845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" event={"ID":"cce091bc-867c-422e-8a3a-0af53ae45b93","Type":"ContainerStarted","Data":"d8791481d6d16cbb7682298ed3e1ea0b3f810cc270ff26fdf9c58395df3b4a2f"}
Jul 06 23:58:33.557958 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.557934    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:33.578127 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.578077    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" podStartSLOduration=76.578061697 podStartE2EDuration="1m16.578061697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:33.576262752 +0000 UTC m=+98.901266049" watchObservedRunningTime="2026-07-06 23:58:33.578061697 +0000 UTC m=+98.903064981"
Jul 06 23:58:33.581027 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:33.581005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:35.568640 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:35.568607    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrhjm" event={"ID":"9745e43d-f62d-476a-980f-54ad7ccf8c8f","Type":"ContainerStarted","Data":"6d5b5467d5b6352c79621061d3359b7f03a15a3304fbbab5ae08e19182240758"}
Jul 06 23:58:35.569020 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:35.568657    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrhjm" event={"ID":"9745e43d-f62d-476a-980f-54ad7ccf8c8f","Type":"ContainerStarted","Data":"2a9f455dec1591bf3bf4550b28db50168bd35e605a6854a6474de57a699d5ebf"}
Jul 06 23:58:35.569020 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:35.568699    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:35.586063 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:35.586016    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qrhjm" podStartSLOduration=65.976330229 podStartE2EDuration="1m7.586001591s" totalImagesPullingTime="1.609671362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:35.584930662 +0000 UTC m=+100.909933945" watchObservedRunningTime="2026-07-06 23:58:35.586001591 +0000 UTC m=+100.911004853"
Jul 06 23:58:36.304611 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:36.304563    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:36.907801 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:36.907768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:36.968128 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:36.968098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:37.447565 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:37.447537    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fxpmb"
Jul 06 23:58:37.502138 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:37.502109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:37.558532 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:37.558500    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:37.923229 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:37.923189    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:58:38.011265 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.011234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.109518 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.109479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.322605 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.322521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.559019 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.558984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.614974 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.614894    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rv95b"]
Jul 06 23:58:38.636504 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.636469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.642605 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.642586    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.645170 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.645151    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 23:58:38.645523 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.645506    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 23:58:38.645647 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.645506    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 23:58:38.646336 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.646317    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 23:58:38.646562 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.646544    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r62mn\""
Jul 06 23:58:38.692922 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.692888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-metrics-client-ca\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.692922 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.692922    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-textfile\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.692949    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-sys\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.692975    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.693028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.693071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-root\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.693100    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4smrv\" (UniqueName: \"kubernetes.io/projected/3985d385-b607-4e6e-9c97-cca5fdca768f-kube-api-access-4smrv\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.693117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-wtmp\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.693152 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.693137    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.793887 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.793855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-sys\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.793887 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.793890    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.793910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.793961    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-sys\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794030    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-root\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:38.794038    2583 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794085    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4smrv\" (UniqueName: \"kubernetes.io/projected/3985d385-b607-4e6e-9c97-cca5fdca768f-kube-api-access-4smrv\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794119 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:58:38.794097    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls podName:3985d385-b607-4e6e-9c97-cca5fdca768f nodeName:}" failed. No retries permitted until 2026-07-06 23:58:39.294079567 +0000 UTC m=+104.619082829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls") pod "node-exporter-rv95b" (UID: "3985d385-b607-4e6e-9c97-cca5fdca768f") : secret "node-exporter-tls" not found
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794127    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-wtmp\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794142    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-root\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794208    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-metrics-client-ca\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-textfile\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794461 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794259    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-wtmp\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794804 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794623    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-textfile\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794804 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794734    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-metrics-client-ca\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.794804 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.794774    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-accelerators-collector-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.796348 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.796309    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.805771 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.805747    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4smrv\" (UniqueName: \"kubernetes.io/projected/3985d385-b607-4e6e-9c97-cca5fdca768f-kube-api-access-4smrv\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:38.839026 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.838995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:38.964852 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:38.964812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.065144 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.065105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.167957 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.167921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.298812 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.298727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:39.301196 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.301158    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3985d385-b607-4e6e-9c97-cca5fdca768f-node-exporter-tls\") pod \"node-exporter-rv95b\" (UID: \"3985d385-b607-4e6e-9c97-cca5fdca768f\") " pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:39.314161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.314129    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.413957 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.413918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.549840 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.549742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.551517 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.551500    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rv95b"
Jul 06 23:58:39.593824 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:39.593798    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3985d385_b607_4e6e_9c97_cca5fdca768f.slice/crio-bdedc09da12cf833417f2bac2e4e47dacbbef99cecd248c316652d84621b9a7c WatchSource:0}: Error finding container bdedc09da12cf833417f2bac2e4e47dacbbef99cecd248c316652d84621b9a7c: Status 404 returned error can't find the container with id bdedc09da12cf833417f2bac2e4e47dacbbef99cecd248c316652d84621b9a7c
Jul 06 23:58:39.659813 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.659777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:39.935264 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:39.935224    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:40.030428 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:40.030391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:40.586633 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:40.586592    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv95b" event={"ID":"3985d385-b607-4e6e-9c97-cca5fdca768f","Type":"ContainerStarted","Data":"bdedc09da12cf833417f2bac2e4e47dacbbef99cecd248c316652d84621b9a7c"}
Jul 06 23:58:41.497679 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.497643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:41.592065 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.592026    2583 generic.go:356] "Generic (PLEG): container finished" podID="3985d385-b607-4e6e-9c97-cca5fdca768f" containerID="ab5ac5c6f5c264a70b47e6e72374abfad78cb9fd675cded4e48743f59715a3df" exitCode=0
Jul 06 23:58:41.592229 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.592115    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv95b" event={"ID":"3985d385-b607-4e6e-9c97-cca5fdca768f","Type":"ContainerDied","Data":"ab5ac5c6f5c264a70b47e6e72374abfad78cb9fd675cded4e48743f59715a3df"}
Jul 06 23:58:41.597870 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.597845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:41.698743 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.698703    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:41.799161 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:41.799069    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:42.058955 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:42.058854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:42.165915 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:42.165882    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:42.597441 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:42.597401    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv95b" event={"ID":"3985d385-b607-4e6e-9c97-cca5fdca768f","Type":"ContainerStarted","Data":"42dfa3aeabdfd01a6960b89955bd26c2100019c21a42ccdc005335da0fd2c29d"}
Jul 06 23:58:42.597855 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:42.597462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rv95b" event={"ID":"3985d385-b607-4e6e-9c97-cca5fdca768f","Type":"ContainerStarted","Data":"887385f818946c5613fcc368677e9c5114d841de8cde8ac1a248930cd5737002"}
Jul 06 23:58:42.617134 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:42.617077    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rv95b" podStartSLOduration=3.017019014 podStartE2EDuration="4.617060712s" totalImagesPullingTime="1.500632822s" totalInitContainerRuntime="99.408876ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:42.616763691 +0000 UTC m=+107.941766975" watchObservedRunningTime="2026-07-06 23:58:42.617060712 +0000 UTC m=+107.942064036"
Jul 06 23:58:43.324413 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:43.324378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.396136 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:43.396100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.525612 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:43.525574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.626621 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:43.626593    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:43.761076 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:43.761033    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:44.155141 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.155102    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:44.664891 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.664860    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 06 23:58:44.667108 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.667093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.669726 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.669705    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 23:58:44.669826 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.669735    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 23:58:44.670669 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.670640    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 23:58:44.670765 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.670711    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-whz66\""
Jul 06 23:58:44.670765 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.670640    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 23:58:44.670885 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.670877    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 23:58:44.670987 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.670970    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 23:58:44.671039 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.671019    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 23:58:44.675276 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.675253    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 23:58:44.677657 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.677636    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 06 23:58:44.750593 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750764 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750764 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk54b\" (UniqueName: \"kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750764 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750711    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750764 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750745    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750897 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750800    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.750897 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.750819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.815220 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.815186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:44.852072 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk54b\" (UniqueName: \"kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852215    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852267 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.852552 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.852280    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.853384 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.853092    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.853384 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.853341    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.853776 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.853742    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.853776 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.853760    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.855287 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.855264    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.855447 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.855430    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.863713 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.863693    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk54b\" (UniqueName: \"kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b\") pod \"console-7d4b64865f-kkjg9\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") " pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:44.976675 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:44.976577    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:45.103720 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:45.103692    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 06 23:58:45.105851 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:45.105819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:45.106471 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:58:45.106446    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcef0e0ea_5160_4d8e_8437_67196d1d2ad5.slice/crio-c9d4f52dd76cef2428abb4490cd0aa22144141457a891b2836b1d763d85723e8 WatchSource:0}: Error finding container c9d4f52dd76cef2428abb4490cd0aa22144141457a891b2836b1d763d85723e8: Status 404 returned error can't find the container with id c9d4f52dd76cef2428abb4490cd0aa22144141457a891b2836b1d763d85723e8
Jul 06 23:58:45.167232 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:45.167199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:45.574808 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:45.574777    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qrhjm"
Jul 06 23:58:45.609311 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:45.609256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64865f-kkjg9" event={"ID":"cef0e0ea-5160-4d8e-8437-67196d1d2ad5","Type":"ContainerStarted","Data":"c9d4f52dd76cef2428abb4490cd0aa22144141457a891b2836b1d763d85723e8"}
Jul 06 23:58:47.155755 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:47.155664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:47.241279 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:47.241240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:48.622315 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:48.622256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64865f-kkjg9" event={"ID":"cef0e0ea-5160-4d8e-8437-67196d1d2ad5","Type":"ContainerStarted","Data":"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"}
Jul 06 23:58:48.639265 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:48.639208    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d4b64865f-kkjg9" podStartSLOduration=1.972106464 podStartE2EDuration="4.63919106s" totalImagesPullingTime="2.667084596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:48.639027558 +0000 UTC m=+113.964030843" watchObservedRunningTime="2026-07-06 23:58:48.63919106 +0000 UTC m=+113.964194344"
Jul 06 23:58:51.137573 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:51.137530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:51.237909 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:51.237871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:54.190770 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.190736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:54.245905 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.245862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:54.711648 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.711618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:54.812834 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.812800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:54.976793 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.976704    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:54.976793 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.976767    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:54.981688 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:54.981665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:55.649955 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:55.649920    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 06 23:58:55.818676 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:55.818643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:55.918944 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:55.918863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:56.020387 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:56.020341    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:58:57.929446 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:57.929416    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:58:59.660603 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:59.660573    2583 generic.go:356] "Generic (PLEG): container finished" podID="699771fc-b797-47a3-9c37-81b0ebbfb0dd" containerID="371849832b6ed9cc1a52279dc25af310bd36c92932e97425493798dc652af791" exitCode=0
Jul 06 23:58:59.660954 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:59.660642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6" event={"ID":"699771fc-b797-47a3-9c37-81b0ebbfb0dd","Type":"ContainerDied","Data":"371849832b6ed9cc1a52279dc25af310bd36c92932e97425493798dc652af791"}
Jul 06 23:58:59.660989 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:58:59.660966    2583 scope.go:117] "RemoveContainer" containerID="371849832b6ed9cc1a52279dc25af310bd36c92932e97425493798dc652af791"
Jul 06 23:59:00.665876 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:00.665844    2583 generic.go:356] "Generic (PLEG): container finished" podID="399f91a1-6840-42c2-b390-bb3e18779244" containerID="4bd4aa766a28c8bdbededb28017c930c27490cc3bd8918bd3a76fe6fbb2cc4e7" exitCode=0
Jul 06 23:59:00.666334 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:00.665932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq" event={"ID":"399f91a1-6840-42c2-b390-bb3e18779244","Type":"ContainerDied","Data":"4bd4aa766a28c8bdbededb28017c930c27490cc3bd8918bd3a76fe6fbb2cc4e7"}
Jul 06 23:59:00.666420 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:00.666343    2583 scope.go:117] "RemoveContainer" containerID="4bd4aa766a28c8bdbededb28017c930c27490cc3bd8918bd3a76fe6fbb2cc4e7"
Jul 06 23:59:00.667908 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:00.667883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-4ckw6" event={"ID":"699771fc-b797-47a3-9c37-81b0ebbfb0dd","Type":"ContainerStarted","Data":"a65ab12d13f0653e2ee5cf58279d33d783e4a6dd08414cbc00919a3f2fd2041f"}
Jul 06 23:59:01.581961 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:01.581899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:01.674360 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:01.674323    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-ztxlq" event={"ID":"399f91a1-6840-42c2-b390-bb3e18779244","Type":"ContainerStarted","Data":"5d0c57a801a42371c52074f6cbead51bf96de0618e68ce7b5bd8cca293c89332"}
Jul 06 23:59:01.686016 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:01.685985    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:01.782615 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:01.782548    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:02.943085 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:02.943042    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" podUID="cce091bc-867c-422e-8a3a-0af53ae45b93" containerName="registry" containerID="cri-o://720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f" gracePeriod=30
Jul 06 23:59:03.190457 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.190436    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:59:03.309472 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309394    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309472 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309443    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309654 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309474    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwqwv\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309654 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309576    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309654 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309621    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309654 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309649    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309831 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309667    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.309831 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309686    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates\") pod \"cce091bc-867c-422e-8a3a-0af53ae45b93\" (UID: \"cce091bc-867c-422e-8a3a-0af53ae45b93\") "
Jul 06 23:59:03.310004 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.309948    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:03.310336 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.310275    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 23:59:03.311922 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.311873    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:03.312028 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.311978    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 23:59:03.312090 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.312024    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:59:03.312142 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.312126    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv" (OuterVolumeSpecName: "kube-api-access-xwqwv") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "kube-api-access-xwqwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:59:03.312327 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.312309    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 23:59:03.318844 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.318816    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cce091bc-867c-422e-8a3a-0af53ae45b93" (UID: "cce091bc-867c-422e-8a3a-0af53ae45b93"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 23:59:03.410653 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410601    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-image-registry-private-configuration\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410653 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410646    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-tls\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410653 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410658    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwqwv\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-kube-api-access-xwqwv\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410653 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410667    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-trusted-ca\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410919 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410677    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cce091bc-867c-422e-8a3a-0af53ae45b93-ca-trust-extracted\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410919 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410685    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cce091bc-867c-422e-8a3a-0af53ae45b93-installation-pull-secrets\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410919 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410693    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cce091bc-867c-422e-8a3a-0af53ae45b93-bound-sa-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.410919 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.410703    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cce091bc-867c-422e-8a3a-0af53ae45b93-registry-certificates\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 06 23:59:03.682535 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.682500    2583 generic.go:356] "Generic (PLEG): container finished" podID="cce091bc-867c-422e-8a3a-0af53ae45b93" containerID="720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f" exitCode=0
Jul 06 23:59:03.682695 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.682549    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-f9b979749-tkpfg"
Jul 06 23:59:03.682695 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.682593    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" event={"ID":"cce091bc-867c-422e-8a3a-0af53ae45b93","Type":"ContainerDied","Data":"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"}
Jul 06 23:59:03.682695 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.682659    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-f9b979749-tkpfg" event={"ID":"cce091bc-867c-422e-8a3a-0af53ae45b93","Type":"ContainerDied","Data":"d8791481d6d16cbb7682298ed3e1ea0b3f810cc270ff26fdf9c58395df3b4a2f"}
Jul 06 23:59:03.682695 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.682687    2583 scope.go:117] "RemoveContainer" containerID="720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"
Jul 06 23:59:03.691083 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.691066    2583 scope.go:117] "RemoveContainer" containerID="720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"
Jul 06 23:59:03.691386 ip-10-0-142-92 kubenswrapper[2583]: E0706 23:59:03.691362    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f\": container with ID starting with 720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f not found: ID does not exist" containerID="720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"
Jul 06 23:59:03.691464 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.691391    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f"} err="failed to get container status \"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f\": rpc error: code = NotFound desc = could not find container \"720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f\": container with ID starting with 720b6c04cbbf2fac96f325f1dab4a83d2310a7df0939f3ca0d55c4a21db92f1f not found: ID does not exist"
Jul 06 23:59:03.704022 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.703996    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:59:03.707680 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:03.707657    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-f9b979749-tkpfg"]
Jul 06 23:59:05.023124 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.023090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:59:05.025524 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.025499    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/91a84a2f-d2eb-419f-b778-bab4fe49a581-metrics-certs\") pod \"network-metrics-daemon-j6rtl\" (UID: \"91a84a2f-d2eb-419f-b778-bab4fe49a581\") " pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:59:05.156936 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.156905    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cce091bc-867c-422e-8a3a-0af53ae45b93" path="/var/lib/kubelet/pods/cce091bc-867c-422e-8a3a-0af53ae45b93/volumes"
Jul 06 23:59:05.173809 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.173778    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wdx8b\""
Jul 06 23:59:05.182140 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.182098    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j6rtl"
Jul 06 23:59:05.314043 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.313926    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j6rtl"]
Jul 06 23:59:05.316629 ip-10-0-142-92 kubenswrapper[2583]: W0706 23:59:05.316597    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a84a2f_d2eb_419f_b778_bab4fe49a581.slice/crio-feec8adb9cd4f75f0de046c8e56072b5af04a1509fccaa3ff2415ed4dfd8e0f2 WatchSource:0}: Error finding container feec8adb9cd4f75f0de046c8e56072b5af04a1509fccaa3ff2415ed4dfd8e0f2: Status 404 returned error can't find the container with id feec8adb9cd4f75f0de046c8e56072b5af04a1509fccaa3ff2415ed4dfd8e0f2
Jul 06 23:59:05.693108 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:05.693059    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6rtl" event={"ID":"91a84a2f-d2eb-419f-b778-bab4fe49a581","Type":"ContainerStarted","Data":"feec8adb9cd4f75f0de046c8e56072b5af04a1509fccaa3ff2415ed4dfd8e0f2"}
Jul 06 23:59:06.698658 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:06.698617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6rtl" event={"ID":"91a84a2f-d2eb-419f-b778-bab4fe49a581","Type":"ContainerStarted","Data":"8ce84fb70aed056795a66160d0439a33675d9963ff1b47340c9b1801c8c82d4d"}
Jul 06 23:59:06.699056 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:06.698679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j6rtl" event={"ID":"91a84a2f-d2eb-419f-b778-bab4fe49a581","Type":"ContainerStarted","Data":"9d8fb8670c9beb46089956f2eb5bd7ee9310bf975fc6d430c8c5aee039b8d18e"}
Jul 06 23:59:06.816030 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:06.815992    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:06.917741 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:06.917706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:09.184762 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:09.184727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:09.238596 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:09.238565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:57.904481 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:57.904445    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 06 23:59:58.007871 ip-10-0-142-92 kubenswrapper[2583]: I0706 23:59:58.007830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:08.201182 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.201126    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j6rtl" podStartSLOduration=192.357503305 podStartE2EDuration="3m13.201063817s" totalImagesPullingTime="843.560512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:59:06.715743938 +0000 UTC m=+132.040747499" watchObservedRunningTime="2026-07-07 00:00:08.201063817 +0000 UTC m=+193.526067093"
Jul 07 00:00:08.202672 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.202647    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:00:08.202944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.202928    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce091bc-867c-422e-8a3a-0af53ae45b93" containerName="registry"
Jul 07 00:00:08.203015 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.202946    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce091bc-867c-422e-8a3a-0af53ae45b93" containerName="registry"
Jul 07 00:00:08.203015 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.202990    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce091bc-867c-422e-8a3a-0af53ae45b93" containerName="registry"
Jul 07 00:00:08.207073 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.207053    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.266639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.266606    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:00:08.300487 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300658 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300505    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300658 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300658 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300658 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300637    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.300807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.300710    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lk98r\" (UniqueName: \"kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.328574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.328531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:08.401427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401432    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401675 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401459    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401675 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401781 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401686    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401781 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401753    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lk98r\" (UniqueName: \"kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.401880 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.401820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.402287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.402261    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.402706 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.402682    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.402805 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.402751    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.402805 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.402795    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.404146 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.404124    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.404592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.404570    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.411664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.411641    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lk98r\" (UniqueName: \"kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r\") pod \"console-6b87645799-9kbkh\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") " pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.492113 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.492085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:08.536761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.536729    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:08.667035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.667011    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:00:08.669374 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:00:08.669348    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b031cb_b49a_4110_b99c_3bfeef91f546.slice/crio-40eeaac37f1a3aab7050170a4fa226038f8dea75292886609bb9797678ccfe67 WatchSource:0}: Error finding container 40eeaac37f1a3aab7050170a4fa226038f8dea75292886609bb9797678ccfe67: Status 404 returned error can't find the container with id 40eeaac37f1a3aab7050170a4fa226038f8dea75292886609bb9797678ccfe67
Jul 07 00:00:08.919478 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.919438    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87645799-9kbkh" event={"ID":"39b031cb-b49a-4110-b99c-3bfeef91f546","Type":"ContainerStarted","Data":"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"}
Jul 07 00:00:08.919679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.919489    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87645799-9kbkh" event={"ID":"39b031cb-b49a-4110-b99c-3bfeef91f546","Type":"ContainerStarted","Data":"40eeaac37f1a3aab7050170a4fa226038f8dea75292886609bb9797678ccfe67"}
Jul 07 00:00:08.938196 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:08.938122    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b87645799-9kbkh" podStartSLOduration=0.938107189 podStartE2EDuration="938.107189ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:08.937231638 +0000 UTC m=+194.262234961" watchObservedRunningTime="2026-07-07 00:00:08.938107189 +0000 UTC m=+194.263110472"
Jul 07 00:00:18.537002 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:18.536957    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:18.537450 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:18.537108    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:18.541951 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:18.541928    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:18.959830 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:18.959803    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:00:19.021856 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:19.021815    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 07 00:00:19.226202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:19.226109    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:19.413837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:19.413797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:00:44.044927 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.044870    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d4b64865f-kkjg9" podUID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" containerName="console" containerID="cri-o://18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6" gracePeriod=15
Jul 07 00:00:44.287217 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.287193    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 07 00:00:44.388031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.387994    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388045    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388070    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qk54b\" (UniqueName: \"kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388257    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388280    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388466 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388331    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert\") pod \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\" (UID: \"cef0e0ea-5160-4d8e-8437-67196d1d2ad5\") "
Jul 07 00:00:44.388533 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388510    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:00:44.388690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388662    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config" (OuterVolumeSpecName: "console-config") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:00:44.388824 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388758    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:00:44.388824 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.388773    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca" (OuterVolumeSpecName: "service-ca") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:00:44.390669 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.390635    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:00:44.390752 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.390665    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:00:44.390752 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.390682    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b" (OuterVolumeSpecName: "kube-api-access-qk54b") pod "cef0e0ea-5160-4d8e-8437-67196d1d2ad5" (UID: "cef0e0ea-5160-4d8e-8437-67196d1d2ad5"). InnerVolumeSpecName "kube-api-access-qk54b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:00:44.489431 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489387    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qk54b\" (UniqueName: \"kubernetes.io/projected/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-kube-api-access-qk54b\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489431 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489422    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-oauth-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489431 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489431    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-service-ca\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489431 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489440    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489449    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-trusted-ca-bundle\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489457    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-oauth-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:44.489690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:44.489465    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cef0e0ea-5160-4d8e-8437-67196d1d2ad5-console-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:00:45.046495 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.046462    2583 generic.go:356] "Generic (PLEG): container finished" podID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" containerID="18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6" exitCode=0
Jul 07 00:00:45.046887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.046506    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d4b64865f-kkjg9"
Jul 07 00:00:45.046887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.046546    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64865f-kkjg9" event={"ID":"cef0e0ea-5160-4d8e-8437-67196d1d2ad5","Type":"ContainerDied","Data":"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"}
Jul 07 00:00:45.046887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.046596    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d4b64865f-kkjg9" event={"ID":"cef0e0ea-5160-4d8e-8437-67196d1d2ad5","Type":"ContainerDied","Data":"c9d4f52dd76cef2428abb4490cd0aa22144141457a891b2836b1d763d85723e8"}
Jul 07 00:00:45.046887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.046618    2583 scope.go:117] "RemoveContainer" containerID="18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"
Jul 07 00:00:45.054944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.054926    2583 scope.go:117] "RemoveContainer" containerID="18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"
Jul 07 00:00:45.055225 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:00:45.055205    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6\": container with ID starting with 18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6 not found: ID does not exist" containerID="18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"
Jul 07 00:00:45.055273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.055234    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6"} err="failed to get container status \"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6\": rpc error: code = NotFound desc = could not find container \"18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6\": container with ID starting with 18af2937a324e2e58106c930a8d0f9bab44b01ba2229067ab800d676fd0622e6 not found: ID does not exist"
Jul 07 00:00:45.065952 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.065926    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 07 00:00:45.068731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.068711    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d4b64865f-kkjg9"]
Jul 07 00:00:45.156260 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:45.156229    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" path="/var/lib/kubelet/pods/cef0e0ea-5160-4d8e-8437-67196d1d2ad5/volumes"
Jul 07 00:00:58.060332 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:00:58.060274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:06.686395 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:06.686354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:13.097119 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:13.097078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:13.156330 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:13.156272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:25.120107 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:25.120028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:25.223963 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:25.223917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:39.236650 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:39.236611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:39.406765 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:39.406726    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:40.966389 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:40.966347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:41.152241 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:41.152183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:01:55.041095 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:55.041068    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:01:55.041831 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:55.041815    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:01:55.051378 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:01:55.051355    2583 kubelet.go:1648] "Image garbage collection succeeded"
Jul 07 00:02:06.794522 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.794490    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:02:06.796986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.794782    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" containerName="console"
Jul 07 00:02:06.796986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.794797    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" containerName="console"
Jul 07 00:02:06.796986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.794854    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cef0e0ea-5160-4d8e-8437-67196d1d2ad5" containerName="console"
Jul 07 00:02:06.797872 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.797851    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.807555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.807531    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:02:06.845971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.845936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:06.905052 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.905021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:06.918115 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918197    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918240    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c6jw\" (UniqueName: \"kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.918468 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.918335    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:06.999013 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:06.998975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:07.019130 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019088    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c6jw\" (UniqueName: \"kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019137    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019227    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019253    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.019361 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.019343    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.020078 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.020046    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.020210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.020046    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.020210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.020163    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.020210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.020198    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.021759 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.021732    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.021870 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.021843    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.026800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.026778    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c6jw\" (UniqueName: \"kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw\") pod \"console-568b686dd4-r6rkh\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") " pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.055265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.055180    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:07.107987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.107957    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:07.239911 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.239887    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:02:07.242467 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:02:07.242427    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8667b5e_9f88_43ef_bb7f_1553a9872186.slice/crio-67e5dbdaeece0bb9849a757990fb42b7e83a7dd21638bac49aeec7d4b53e4498 WatchSource:0}: Error finding container 67e5dbdaeece0bb9849a757990fb42b7e83a7dd21638bac49aeec7d4b53e4498: Status 404 returned error can't find the container with id 67e5dbdaeece0bb9849a757990fb42b7e83a7dd21638bac49aeec7d4b53e4498
Jul 07 00:02:07.328704 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.328616    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568b686dd4-r6rkh" event={"ID":"a8667b5e-9f88-43ef-bb7f-1553a9872186","Type":"ContainerStarted","Data":"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"}
Jul 07 00:02:07.328704 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.328674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568b686dd4-r6rkh" event={"ID":"a8667b5e-9f88-43ef-bb7f-1553a9872186","Type":"ContainerStarted","Data":"67e5dbdaeece0bb9849a757990fb42b7e83a7dd21638bac49aeec7d4b53e4498"}
Jul 07 00:02:07.346224 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.346026    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-568b686dd4-r6rkh" podStartSLOduration=1.3460043449999999 podStartE2EDuration="1.346004345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:07.345998393 +0000 UTC m=+312.671001678" watchObservedRunningTime="2026-07-07 00:02:07.346004345 +0000 UTC m=+312.671007630"
Jul 07 00:02:07.686041 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.686002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:07.745696 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:07.745656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:17.108969 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.108911    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:17.109416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.109019    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:17.113734 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.113711    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:17.367311 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.367274    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:02:17.409929 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.409893    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:02:17.519257 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:17.519217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:18.972885 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:18.972853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:22.534811 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.534771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:22.626120 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.626068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:22.632959 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.632927    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"]
Jul 07 00:02:22.636450 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.636436    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.638976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.638955    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 07 00:02:22.638976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.638973    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 07 00:02:22.640093 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.640069    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 07 00:02:22.640222 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.640096    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 07 00:02:22.640222 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.640105    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 07 00:02:22.640222 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.640095    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 07 00:02:22.640222 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.640072    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 07 00:02:22.648855 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.648828    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"]
Jul 07 00:02:22.701941 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.701898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:22.746579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.746579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.746791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746618    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.746791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.746791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746742    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4snsg\" (UniqueName: \"kubernetes.io/projected/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-kube-api-access-4snsg\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.746897 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.746831    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847438 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847346    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847438 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847385    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847438 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847404    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847438 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847438    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847559    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4snsg\" (UniqueName: \"kubernetes.io/projected/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-kube-api-access-4snsg\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.847749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.847645    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.848449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.848418    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.850006 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.849976    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.850105 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.849976    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-ca\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.850147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.850125    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.850310 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.850269    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.855592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.855568    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4snsg\" (UniqueName: \"kubernetes.io/projected/9ce1101d-9345-4eda-9e98-4ce631cfcf5b-kube-api-access-4snsg\") pod \"cluster-proxy-proxy-agent-77f7798f57-vlndp\" (UID: \"9ce1101d-9345-4eda-9e98-4ce631cfcf5b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:22.956424 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:22.956389    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"
Jul 07 00:02:23.103795 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:23.103704    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp"]
Jul 07 00:02:23.106727 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:02:23.106703    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ce1101d_9345_4eda_9e98_4ce631cfcf5b.slice/crio-888a4f5d1ee328d8dbdd09cd210a84d8ef97f6b410500f7142f04b9693361c22 WatchSource:0}: Error finding container 888a4f5d1ee328d8dbdd09cd210a84d8ef97f6b410500f7142f04b9693361c22: Status 404 returned error can't find the container with id 888a4f5d1ee328d8dbdd09cd210a84d8ef97f6b410500f7142f04b9693361c22
Jul 07 00:02:23.108816 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:23.108792    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:02:23.386237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:23.386185    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp" event={"ID":"9ce1101d-9345-4eda-9e98-4ce631cfcf5b","Type":"ContainerStarted","Data":"888a4f5d1ee328d8dbdd09cd210a84d8ef97f6b410500f7142f04b9693361c22"}
Jul 07 00:02:25.051495 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:25.051401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:25.150552 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:25.150420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:28.411134 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:28.411100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp" event={"ID":"9ce1101d-9345-4eda-9e98-4ce631cfcf5b","Type":"ContainerStarted","Data":"89d9d872b9b6416e8830c1833a09110500d85c5bbf7d66fc98de284a7049fd5d"}
Jul 07 00:02:28.411551 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:28.411156    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp" event={"ID":"9ce1101d-9345-4eda-9e98-4ce631cfcf5b","Type":"ContainerStarted","Data":"0c041ecc54931c211ed8e30981181e68f6fc29d7e7b2e4321be7bc6104122f3d"}
Jul 07 00:02:28.411551 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:28.411175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp" event={"ID":"9ce1101d-9345-4eda-9e98-4ce631cfcf5b","Type":"ContainerStarted","Data":"9b9b6014e924e374c332f8b95864d9ba2bf8e335679c2633a8e6aaf039cdee97"}
Jul 07 00:02:28.428644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:28.428588    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-77f7798f57-vlndp" podStartSLOduration=2.153453565 podStartE2EDuration="6.428572604s" totalImagesPullingTime="4.275119039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:02:28.427591894 +0000 UTC m=+333.752595180" watchObservedRunningTime="2026-07-07 00:02:28.428572604 +0000 UTC m=+333.753575888"
Jul 07 00:02:33.952545 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:33.952511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:42.437104 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.437060    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b87645799-9kbkh" podUID="39b031cb-b49a-4110-b99c-3bfeef91f546" containerName="console" containerID="cri-o://1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2" gracePeriod=15
Jul 07 00:02:42.682598 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.682572    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:02:42.822654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822556    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822609    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822644    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822683    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822732    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lk98r\" (UniqueName: \"kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822771    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.822946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.822797    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle\") pod \"39b031cb-b49a-4110-b99c-3bfeef91f546\" (UID: \"39b031cb-b49a-4110-b99c-3bfeef91f546\") "
Jul 07 00:02:42.823157 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.823100    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:02:42.823237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.823209    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca" (OuterVolumeSpecName: "service-ca") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:02:42.823382 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.823357    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config" (OuterVolumeSpecName: "console-config") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:02:42.823455 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.823376    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:02:42.824779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.824756    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:02:42.824859 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.824812    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:02:42.824899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.824869    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r" (OuterVolumeSpecName: "kube-api-access-lk98r") pod "39b031cb-b49a-4110-b99c-3bfeef91f546" (UID: "39b031cb-b49a-4110-b99c-3bfeef91f546"). InnerVolumeSpecName "kube-api-access-lk98r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:02:42.923697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923657    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lk98r\" (UniqueName: \"kubernetes.io/projected/39b031cb-b49a-4110-b99c-3bfeef91f546-kube-api-access-lk98r\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923693    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-console-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923702    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-trusted-ca-bundle\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923934 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923712    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923934 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923721    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39b031cb-b49a-4110-b99c-3bfeef91f546-console-oauth-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923934 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923729    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-oauth-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:42.923934 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:42.923737    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39b031cb-b49a-4110-b99c-3bfeef91f546-service-ca\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:02:43.463987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.463894    2583 generic.go:356] "Generic (PLEG): container finished" podID="39b031cb-b49a-4110-b99c-3bfeef91f546" containerID="1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2" exitCode=0
Jul 07 00:02:43.463987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.463934    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b87645799-9kbkh"
Jul 07 00:02:43.463987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.463971    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87645799-9kbkh" event={"ID":"39b031cb-b49a-4110-b99c-3bfeef91f546","Type":"ContainerDied","Data":"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"}
Jul 07 00:02:43.464600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.464014    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b87645799-9kbkh" event={"ID":"39b031cb-b49a-4110-b99c-3bfeef91f546","Type":"ContainerDied","Data":"40eeaac37f1a3aab7050170a4fa226038f8dea75292886609bb9797678ccfe67"}
Jul 07 00:02:43.464600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.464035    2583 scope.go:117] "RemoveContainer" containerID="1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"
Jul 07 00:02:43.472138 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.472122    2583 scope.go:117] "RemoveContainer" containerID="1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"
Jul 07 00:02:43.472421 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:02:43.472405    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2\": container with ID starting with 1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2 not found: ID does not exist" containerID="1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"
Jul 07 00:02:43.472469 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.472429    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2"} err="failed to get container status \"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2\": rpc error: code = NotFound desc = could not find container \"1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2\": container with ID starting with 1a8af5087a5324d2bb7f374e0f7883b1f3543cb42ffb19e2169ee0c64411daf2 not found: ID does not exist"
Jul 07 00:02:43.479608 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.479585    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:02:43.486213 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:43.486190    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b87645799-9kbkh"]
Jul 07 00:02:45.156699 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:45.156665    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b031cb-b49a-4110-b99c-3bfeef91f546" path="/var/lib/kubelet/pods/39b031cb-b49a-4110-b99c-3bfeef91f546/volumes"
Jul 07 00:02:52.024238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:52.024197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:53.413861 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:53.413828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:55.877501 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:55.877464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:55.976445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:55.976409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:57.609511 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:57.609471    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:57.718620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:57.718580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:58.434976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:58.434936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:58.768253 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:58.768169    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:02:59.742051 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:02:59.742009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:00.087845 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:00.087761    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:08.485749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:08.485711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:09.235562 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:09.235525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:09.887558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:09.887521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:16.066991 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:16.066948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:16.229639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:16.229608    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:18.655330 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.655284    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k4drm"]
Jul 07 00:03:18.655714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.655553    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b031cb-b49a-4110-b99c-3bfeef91f546" containerName="console"
Jul 07 00:03:18.655714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.655563    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b031cb-b49a-4110-b99c-3bfeef91f546" containerName="console"
Jul 07 00:03:18.655714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.655611    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b031cb-b49a-4110-b99c-3bfeef91f546" containerName="console"
Jul 07 00:03:18.657388 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.657363    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.660890 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.660868    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 07 00:03:18.660981 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.660895    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-m2gn7\""
Jul 07 00:03:18.661908 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.661894    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 07 00:03:18.675895 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.675869    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k4drm"]
Jul 07 00:03:18.784879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.784842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-bound-sa-token\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.784879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.784885    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j95df\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-kube-api-access-j95df\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.886241 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.886199    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j95df\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-kube-api-access-j95df\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.886471 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.886359    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-bound-sa-token\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.899745 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.899712    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j95df\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-kube-api-access-j95df\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.899875 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.899713    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/143f3d67-8ae0-4f8e-8a46-52ab4fb285c1-bound-sa-token\") pod \"cert-manager-55b4d954d5-k4drm\" (UID: \"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1\") " pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:18.965825 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:18.965738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-k4drm"
Jul 07 00:03:19.096376 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:19.096341    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-k4drm"]
Jul 07 00:03:19.100746 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:03:19.100715    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod143f3d67_8ae0_4f8e_8a46_52ab4fb285c1.slice/crio-30216d0788c312b3e4479f72d4574032e59d5361dabec0ea737210f28579429c WatchSource:0}: Error finding container 30216d0788c312b3e4479f72d4574032e59d5361dabec0ea737210f28579429c: Status 404 returned error can't find the container with id 30216d0788c312b3e4479f72d4574032e59d5361dabec0ea737210f28579429c
Jul 07 00:03:19.600262 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:19.600217    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k4drm" event={"ID":"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1","Type":"ContainerStarted","Data":"30216d0788c312b3e4479f72d4574032e59d5361dabec0ea737210f28579429c"}
Jul 07 00:03:22.344626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:22.344479    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:22.521019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:22.520978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:22.612806 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:22.612723    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-k4drm" event={"ID":"143f3d67-8ae0-4f8e-8a46-52ab4fb285c1","Type":"ContainerStarted","Data":"547834582c1e56d94f4b3335b3f8d77e9a11e74f0ab21cd5303a32450ef3bff5"}
Jul 07 00:03:24.108819 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:24.108785    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.646985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:28.646947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:28.773073 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:28.773030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:40.720845 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:40.720810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:40.865328 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:40.865279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:41.345592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.345562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:41.858867 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.858814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:41.970451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.970397    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-k4drm" podStartSLOduration=21.162860759 podStartE2EDuration="23.970381648s" totalImagesPullingTime="2.807520889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:22.660700578 +0000 UTC m=+387.985703863" watchObservedRunningTime="2026-07-07 00:03:41.970381648 +0000 UTC m=+407.295384931"
Jul 07 00:03:41.971218 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.971200    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"]
Jul 07 00:03:41.974458 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.974439    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:41.977838 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.977818    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 07 00:03:41.978280 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.978260    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 07 00:03:41.978506 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.978494    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 07 00:03:41.978551 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.978517    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 07 00:03:41.979703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.979686    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dmrqj\""
Jul 07 00:03:41.997752 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:41.997722    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"]
Jul 07 00:03:42.003906 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.003869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.077476 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.077435    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.077476 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.077473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.077695 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.077540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.077695 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.077563    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8kx4\" (UniqueName: \"kubernetes.io/projected/196299b4-a3e0-4c6f-92c5-d3125246886c-kube-api-access-l8kx4\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.077695 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.077595    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.178631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.178586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.178631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.178631    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.178902 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.178687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.178902 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.178712    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8kx4\" (UniqueName: \"kubernetes.io/projected/196299b4-a3e0-4c6f-92c5-d3125246886c-kube-api-access-l8kx4\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.178902 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.178850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.179128 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.179107    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-manifests\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.181095 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.181074    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/196299b4-a3e0-4c6f-92c5-d3125246886c-tmp\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.181210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.181190    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.181253 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.181215    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/196299b4-a3e0-4c6f-92c5-d3125246886c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.195095 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.195061    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8kx4\" (UniqueName: \"kubernetes.io/projected/196299b4-a3e0-4c6f-92c5-d3125246886c-kube-api-access-l8kx4\") pod \"opendatahub-operator-controller-manager-6b6d6b477c-zhxp2\" (UID: \"196299b4-a3e0-4c6f-92c5-d3125246886c\") " pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.285052 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.285023    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:42.295411 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.295369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.472406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.472320    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"]
Jul 07 00:03:42.475222 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:03:42.475194    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod196299b4_a3e0_4c6f_92c5_d3125246886c.slice/crio-64798d744d8ae3daca17c752e14a10911a57a2105d1a15ba71d96094371f13e5 WatchSource:0}: Error finding container 64798d744d8ae3daca17c752e14a10911a57a2105d1a15ba71d96094371f13e5: Status 404 returned error can't find the container with id 64798d744d8ae3daca17c752e14a10911a57a2105d1a15ba71d96094371f13e5
Jul 07 00:03:42.551310 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.551268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.676789 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.676755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.683040 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.683005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2" event={"ID":"196299b4-a3e0-4c6f-92c5-d3125246886c","Type":"ContainerStarted","Data":"64798d744d8ae3daca17c752e14a10911a57a2105d1a15ba71d96094371f13e5"}
Jul 07 00:03:42.796435 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.796336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:42.929262 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:42.929227    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:43.027875 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:43.027833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:43.132835 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:43.132724    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:43.774624 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:43.774570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:45.697443 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:45.697408    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2" event={"ID":"196299b4-a3e0-4c6f-92c5-d3125246886c","Type":"ContainerStarted","Data":"4849025856ba95c02b4cf4eb737c08a51841904e8a76bd4a334f04c1124fae0e"}
Jul 07 00:03:46.708581 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:46.708494    2583 generic.go:356] "Generic (PLEG): container finished" podID="196299b4-a3e0-4c6f-92c5-d3125246886c" containerID="4849025856ba95c02b4cf4eb737c08a51841904e8a76bd4a334f04c1124fae0e" exitCode=0
Jul 07 00:03:46.708581 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:46.708566    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2" event={"ID":"196299b4-a3e0-4c6f-92c5-d3125246886c","Type":"ContainerDied","Data":"4849025856ba95c02b4cf4eb737c08a51841904e8a76bd4a334f04c1124fae0e"}
Jul 07 00:03:47.714763 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:47.714726    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2" event={"ID":"196299b4-a3e0-4c6f-92c5-d3125246886c","Type":"ContainerStarted","Data":"673f0c6ec1acd7cb420ca9db22a9ad1ac97b6d1ad072cd8a7b52b96c06ceaa59"}
Jul 07 00:03:47.715166 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:47.714825    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:03:47.750313 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:47.750235    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2" podStartSLOduration=2.439047645 podStartE2EDuration="6.750218636s" totalImagesPullingTime="3.281346612s" totalInitContainerRuntime="1.029824379s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:03:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:03:47.748211206 +0000 UTC m=+413.073214489" watchObservedRunningTime="2026-07-07 00:03:47.750218636 +0000 UTC m=+413.075221921"
Jul 07 00:03:53.437578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:53.437533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:53.534093 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:53.534060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:03:58.721848 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:03:58.721818    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b6d6b477c-zhxp2"
Jul 07 00:04:01.777644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:01.777599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:01.934010 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:01.933963    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:02.606370 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.606333    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9d52g"]
Jul 07 00:04:02.613239 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.613214    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.615962 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.615940    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 07 00:04:02.616042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.615946    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-z8tbk\""
Jul 07 00:04:02.622307 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.622266    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9d52g"]
Jul 07 00:04:02.643974 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.643931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhw4f\" (UniqueName: \"kubernetes.io/projected/d53f54a4-54a3-405d-8987-a4ca486b9991-kube-api-access-zhw4f\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.643974 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.643979    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.684758 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.684715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:02.744633 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.744587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhw4f\" (UniqueName: \"kubernetes.io/projected/d53f54a4-54a3-405d-8987-a4ca486b9991-kube-api-access-zhw4f\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.744814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.744642    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.744865 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:04:02.744827    2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 07 00:04:02.744931 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:04:02.744919    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert podName:d53f54a4-54a3-405d-8987-a4ca486b9991 nodeName:}" failed. No retries permitted until 2026-07-07 00:04:03.244894674 +0000 UTC m=+428.569897938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert") pod "odh-model-controller-7998c67dc4-9d52g" (UID: "d53f54a4-54a3-405d-8987-a4ca486b9991") : secret "odh-model-controller-webhook-cert" not found
Jul 07 00:04:02.760563 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.760532    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhw4f\" (UniqueName: \"kubernetes.io/projected/d53f54a4-54a3-405d-8987-a4ca486b9991-kube-api-access-zhw4f\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:02.784882 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.784848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:02.884609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.884577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:02.983106 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:02.983065    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:03.249235 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.249048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:03.251954 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.251926    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d53f54a4-54a3-405d-8987-a4ca486b9991-cert\") pod \"odh-model-controller-7998c67dc4-9d52g\" (UID: \"d53f54a4-54a3-405d-8987-a4ca486b9991\") " pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:03.302093 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.302059    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kqksn"]
Jul 07 00:04:03.304327 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.304285    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.307682 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.307658    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-vhrpz\""
Jul 07 00:04:03.307777 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.307717    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 07 00:04:03.319840 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.319817    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kqksn"]
Jul 07 00:04:03.350573 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.350529    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhd4b\" (UniqueName: \"kubernetes.io/projected/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-kube-api-access-zhd4b\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.350764 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.350582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.360278 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.360248    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:03.451650 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.451598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhd4b\" (UniqueName: \"kubernetes.io/projected/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-kube-api-access-zhd4b\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.451650 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.451669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.454184 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.454155    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-tls-certs\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.464648 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.464620    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhd4b\" (UniqueName: \"kubernetes.io/projected/31a32a62-84c5-4cec-9b52-ee28f8bfddb3-kube-api-access-zhd4b\") pod \"model-serving-api-54d89bd5fc-kqksn\" (UID: \"31a32a62-84c5-4cec-9b52-ee28f8bfddb3\") " pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.468604 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.468573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:03.524791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.524701    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:03.614711 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.614678    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:03.677844 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.677812    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-9d52g"]
Jul 07 00:04:03.681986 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:03.681948    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53f54a4_54a3_405d_8987_a4ca486b9991.slice/crio-65926b3d0b6fac0dec5bd2112bd8046de9f751ed0ef477e926feb7d390503666 WatchSource:0}: Error finding container 65926b3d0b6fac0dec5bd2112bd8046de9f751ed0ef477e926feb7d390503666: Status 404 returned error can't find the container with id 65926b3d0b6fac0dec5bd2112bd8046de9f751ed0ef477e926feb7d390503666
Jul 07 00:04:03.752586 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.752553    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kqksn"]
Jul 07 00:04:03.756468 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:03.756419    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a32a62_84c5_4cec_9b52_ee28f8bfddb3.slice/crio-a5c3171c28249a2309aaacc40bc4cdd974da205424c102774397512cb11224e6 WatchSource:0}: Error finding container a5c3171c28249a2309aaacc40bc4cdd974da205424c102774397512cb11224e6: Status 404 returned error can't find the container with id a5c3171c28249a2309aaacc40bc4cdd974da205424c102774397512cb11224e6
Jul 07 00:04:03.782734 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.782634    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn" event={"ID":"31a32a62-84c5-4cec-9b52-ee28f8bfddb3","Type":"ContainerStarted","Data":"a5c3171c28249a2309aaacc40bc4cdd974da205424c102774397512cb11224e6"}
Jul 07 00:04:03.783706 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:03.783679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g" event={"ID":"d53f54a4-54a3-405d-8987-a4ca486b9991","Type":"ContainerStarted","Data":"65926b3d0b6fac0dec5bd2112bd8046de9f751ed0ef477e926feb7d390503666"}
Jul 07 00:04:06.799868 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:06.799831    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g" event={"ID":"d53f54a4-54a3-405d-8987-a4ca486b9991","Type":"ContainerStarted","Data":"a017eb6372c70466c88d23c2bcfde9995134ca0b61c2736246feddcf12297869"}
Jul 07 00:04:06.800267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:06.799925    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:06.816747 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:06.816689    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g" podStartSLOduration=2.197875533 podStartE2EDuration="4.816673375s" totalImagesPullingTime="2.618797842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:06.815766801 +0000 UTC m=+432.140770085" watchObservedRunningTime="2026-07-07 00:04:06.816673375 +0000 UTC m=+432.141676690"
Jul 07 00:04:07.805947 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.805683    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn" event={"ID":"31a32a62-84c5-4cec-9b52-ee28f8bfddb3","Type":"ContainerStarted","Data":"e2894d192b7e4d1a70d232a344e24112e45f331be76794ffe000d67a7cfaac71"}
Jul 07 00:04:07.816532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.816501    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:07.824663 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.824617    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn" podStartSLOduration=1.090927157 podStartE2EDuration="4.824598136s" totalImagesPullingTime="3.733670979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:07.822259971 +0000 UTC m=+433.147263254" watchObservedRunningTime="2026-07-07 00:04:07.824598136 +0000 UTC m=+433.149601420"
Jul 07 00:04:07.967601 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.967565    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"]
Jul 07 00:04:07.969896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.969873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:07.972671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.972640    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 07 00:04:07.972785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.972720    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tdzqc\""
Jul 07 00:04:07.973372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.973355    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 07 00:04:07.993973 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.993939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmd8k\" (UniqueName: \"kubernetes.io/projected/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-kube-api-access-tmd8k\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:07.993973 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.993978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tmp\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:07.994186 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:07.994054    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.000387 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.000351    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"]
Jul 07 00:04:08.095016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.094918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmd8k\" (UniqueName: \"kubernetes.io/projected/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-kube-api-access-tmd8k\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.095016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.094967    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tmp\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.095016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.095018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.095256 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:04:08.095138    2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 07 00:04:08.095256 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:04:08.095217    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs podName:f65a97a0-2704-48f5-a88a-6f8e8b32cf2d nodeName:}" failed. No retries permitted until 2026-07-07 00:04:08.595194456 +0000 UTC m=+433.920197724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs") pod "kube-auth-proxy-568f9c9968-m68pf" (UID: "f65a97a0-2704-48f5-a88a-6f8e8b32cf2d") : secret "kube-auth-proxy-tls" not found
Jul 07 00:04:08.097466 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.097432    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tmp\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.103716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.103690    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmd8k\" (UniqueName: \"kubernetes.io/projected/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-kube-api-access-tmd8k\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.441451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.441413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:08.598252 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.598214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.601386 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.601356    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f65a97a0-2704-48f5-a88a-6f8e8b32cf2d-tls-certs\") pod \"kube-auth-proxy-568f9c9968-m68pf\" (UID: \"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d\") " pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:08.810591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.810497    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:08.878895 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:08.878863    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"
Jul 07 00:04:09.020088 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.020057    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-568f9c9968-m68pf"]
Jul 07 00:04:09.021496 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:09.021472    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf65a97a0_2704_48f5_a88a_6f8e8b32cf2d.slice/crio-b155fba1b567769e5f90dfc44ca1009604598291d81aa8c81310ae40c0872769 WatchSource:0}: Error finding container b155fba1b567769e5f90dfc44ca1009604598291d81aa8c81310ae40c0872769: Status 404 returned error can't find the container with id b155fba1b567769e5f90dfc44ca1009604598291d81aa8c81310ae40c0872769
Jul 07 00:04:09.066493 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.066382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:09.067556 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.067532    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:09.070139 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.070123    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.072649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.072627    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 07 00:04:09.072760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.072733    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 07 00:04:09.072988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.072972    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c2d9q\""
Jul 07 00:04:09.073144 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.073131    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 07 00:04:09.073195 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.073148    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 07 00:04:09.083261 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.083235    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:09.103036 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103007    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103185 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103185 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103152    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103306 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103306 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103222    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfqqb\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103306 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103249    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.103306 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.103269    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.132636 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.132602    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:09.204606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204569    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.204826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204649    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.204826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204692    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.204826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204736    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.204826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfqqb\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.204826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.205059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.204843    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.205842 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.205814    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.207862 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.207800    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.207985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.207868    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.208077 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.208003    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.208166 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.208089    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.213857 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.213835    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfqqb\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.215169 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.215139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token\") pod \"istiod-openshift-gateway-7574459656-8wxk5\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.379977 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.379930    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:09.563902 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.563871    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:09.564537 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:09.564513    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df853ec_2f49_4695_9693_c358c0c1b7fc.slice/crio-da0d5fafee93d6b685d4db2ddedb205f65ce3a60692b176729637fdea0f5f861 WatchSource:0}: Error finding container da0d5fafee93d6b685d4db2ddedb205f65ce3a60692b176729637fdea0f5f861: Status 404 returned error can't find the container with id da0d5fafee93d6b685d4db2ddedb205f65ce3a60692b176729637fdea0f5f861
Jul 07 00:04:09.820877 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.820786    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf" event={"ID":"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d","Type":"ContainerStarted","Data":"b155fba1b567769e5f90dfc44ca1009604598291d81aa8c81310ae40c0872769"}
Jul 07 00:04:09.821985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:09.821959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" event={"ID":"9df853ec-2f49-4695-9693-c358c0c1b7fc","Type":"ContainerStarted","Data":"da0d5fafee93d6b685d4db2ddedb205f65ce3a60692b176729637fdea0f5f861"}
Jul 07 00:04:10.903853 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:10.903805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:12.822035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:12.821997    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:04:12.822320 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:12.822071    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:04:13.842890 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.842855    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf" event={"ID":"f65a97a0-2704-48f5-a88a-6f8e8b32cf2d","Type":"ContainerStarted","Data":"c270898e4a6bfa86ca8f8499229e0840a1558375e33a7de63a6c04c4819b0581"}
Jul 07 00:04:13.844981 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.844950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" event={"ID":"9df853ec-2f49-4695-9693-c358c0c1b7fc","Type":"ContainerStarted","Data":"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"}
Jul 07 00:04:13.845179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.845125    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:13.847121 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.847095    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-8wxk5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:04:13.847238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.847151    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:04:13.900203 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.900154    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-568f9c9968-m68pf" podStartSLOduration=3.156632259 podStartE2EDuration="6.900139537s" totalImagesPullingTime="3.743507278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:13.894773265 +0000 UTC m=+439.219776552" watchObservedRunningTime="2026-07-07 00:04:13.900139537 +0000 UTC m=+439.225142821"
Jul 07 00:04:13.966588 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:13.966529    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" podStartSLOduration=1.711416276 podStartE2EDuration="4.966511244s" totalImagesPullingTime="3.255094968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:13.964526339 +0000 UTC m=+439.289529624" watchObservedRunningTime="2026-07-07 00:04:13.966511244 +0000 UTC m=+439.291514519"
Jul 07 00:04:14.002926 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.002888    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:14.090578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.090538    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"]
Jul 07 00:04:14.093800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.093742    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.096905 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.096871    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-llr4d\""
Jul 07 00:04:14.097147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.097127    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 07 00:04:14.098737 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.098713    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"]
Jul 07 00:04:14.102111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.102093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.105010 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.104989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 07 00:04:14.105371 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.105355    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hlkb4\""
Jul 07 00:04:14.117555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.117528    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"]
Jul 07 00:04:14.122174 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.122147    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"]
Jul 07 00:04:14.131211 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.131182    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:14.148451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.148423    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.148579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.148458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjcm\" (UniqueName: \"kubernetes.io/projected/3e915ee8-6de5-4fd8-90cb-86e82fad7377-kube-api-access-fwjcm\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.148579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.148557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6qk2\" (UniqueName: \"kubernetes.io/projected/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-kube-api-access-x6qk2\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.148681 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.148620    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e915ee8-6de5-4fd8-90cb-86e82fad7377-cert\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.249810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.249774    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6qk2\" (UniqueName: \"kubernetes.io/projected/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-kube-api-access-x6qk2\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.249976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.249837    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e915ee8-6de5-4fd8-90cb-86e82fad7377-cert\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.249976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.249881    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.249976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.249905    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjcm\" (UniqueName: \"kubernetes.io/projected/3e915ee8-6de5-4fd8-90cb-86e82fad7377-kube-api-access-fwjcm\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.252425 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.252393    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.252425 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.252402    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e915ee8-6de5-4fd8-90cb-86e82fad7377-cert\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.262467 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.262439    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6qk2\" (UniqueName: \"kubernetes.io/projected/a5b59986-fc18-4fd7-8bd9-ec3803d58fa0-kube-api-access-x6qk2\") pod \"llmisvc-controller-manager-777c6bd7d5-2lbc5\" (UID: \"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.262583 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.262541    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjcm\" (UniqueName: \"kubernetes.io/projected/3e915ee8-6de5-4fd8-90cb-86e82fad7377-kube-api-access-fwjcm\") pod \"kserve-controller-manager-d6f4b65b8-2vdcm\" (UID: \"3e915ee8-6de5-4fd8-90cb-86e82fad7377\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.282667 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.282634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:14.394767 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.394733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:14.406653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.406600    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:14.416651 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.416626    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:14.573592 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:14.573563    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda5b59986_fc18_4fd7_8bd9_ec3803d58fa0.slice/crio-b5963a090a2ad51ebefdfd80a9dbeff66932f93af0798b01fe20258ec3dc8591 WatchSource:0}: Error finding container b5963a090a2ad51ebefdfd80a9dbeff66932f93af0798b01fe20258ec3dc8591: Status 404 returned error can't find the container with id b5963a090a2ad51ebefdfd80a9dbeff66932f93af0798b01fe20258ec3dc8591
Jul 07 00:04:14.576877 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.576855    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"]
Jul 07 00:04:14.590752 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.590731    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"]
Jul 07 00:04:14.592431 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:14.592403    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e915ee8_6de5_4fd8_90cb_86e82fad7377.slice/crio-7580372b1ba1cbdff2939b1383e190015c4125188c355067896722a00af04120 WatchSource:0}: Error finding container 7580372b1ba1cbdff2939b1383e190015c4125188c355067896722a00af04120: Status 404 returned error can't find the container with id 7580372b1ba1cbdff2939b1383e190015c4125188c355067896722a00af04120
Jul 07 00:04:14.850665 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.850571    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5" event={"ID":"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0","Type":"ContainerStarted","Data":"b5963a090a2ad51ebefdfd80a9dbeff66932f93af0798b01fe20258ec3dc8591"}
Jul 07 00:04:14.856683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.855147    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm" event={"ID":"3e915ee8-6de5-4fd8-90cb-86e82fad7377","Type":"ContainerStarted","Data":"7580372b1ba1cbdff2939b1383e190015c4125188c355067896722a00af04120"}
Jul 07 00:04:14.856683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.855976    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-8wxk5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:04:14.856683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:14.856033    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:04:15.132793 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.132757    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"]
Jul 07 00:04:15.136177 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.136151    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.139816 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.139790    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9swnz\""
Jul 07 00:04:15.163632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.163593    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"]
Jul 07 00:04:15.218200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.218163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:15.258488 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.258446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.258785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.258511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.258785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.258537    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.258785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.258602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.259434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.259403    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.259549 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.259473    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42796cd9-a85e-40ea-9d1a-4bd12addb356-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.259637 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.259612    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl5vk\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-kube-api-access-xl5vk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.259749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.259666    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.259820 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.259776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.319534 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.319483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:15.360509 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360509 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42796cd9-a85e-40ea-9d1a-4bd12addb356-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360775    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl5vk\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-kube-api-access-xl5vk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.360919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.360814    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.361112 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.361026    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.361436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.361398    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.361544 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.361434    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.361544 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.361442    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.361660 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.361641    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42796cd9-a85e-40ea-9d1a-4bd12addb356-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.362716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.362699    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.362995 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.362979    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.371909 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.371882    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.372055 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.372004    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl5vk\" (UniqueName: \"kubernetes.io/projected/42796cd9-a85e-40ea-9d1a-4bd12addb356-kube-api-access-xl5vk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4khb4s\" (UID: \"42796cd9-a85e-40ea-9d1a-4bd12addb356\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.417167 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.417052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:15.449117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.449082    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:15.704256 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.704226    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"]
Jul 07 00:04:15.787626 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:04:15.787588    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42796cd9_a85e_40ea_9d1a_4bd12addb356.slice/crio-29f1b38a582b087d16f9b5e4a766188c955b244cf103fa647f6935ead8854c08 WatchSource:0}: Error finding container 29f1b38a582b087d16f9b5e4a766188c955b244cf103fa647f6935ead8854c08: Status 404 returned error can't find the container with id 29f1b38a582b087d16f9b5e4a766188c955b244cf103fa647f6935ead8854c08
Jul 07 00:04:15.861331 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:15.861270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s" event={"ID":"42796cd9-a85e-40ea-9d1a-4bd12addb356","Type":"ContainerStarted","Data":"29f1b38a582b087d16f9b5e4a766188c955b244cf103fa647f6935ead8854c08"}
Jul 07 00:04:16.117542 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.117504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:16.245191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.245106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:16.450967 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.450884    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:16.451174 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.451093    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery" containerID="cri-o://08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62" gracePeriod=30
Jul 07 00:04:16.451955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.451935    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:16.477250 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:16.477213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:17.077530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.077498    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:17.175037 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.174953    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175037 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175028    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175064    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175089    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175112    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfqqb\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175206    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175267 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175246    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs\") pod \"9df853ec-2f49-4695-9693-c358c0c1b7fc\" (UID: \"9df853ec-2f49-4695-9693-c358c0c1b7fc\") "
Jul 07 00:04:17.175775 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.175701    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:04:17.178336 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.178259    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs" (OuterVolumeSpecName: "local-certs") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:04:17.178336 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.178265    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:04:17.178704 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.178667    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token" (OuterVolumeSpecName: "istio-token") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:17.178886 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.178863    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts" (OuterVolumeSpecName: "cacerts") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:04:17.178979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.178883    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:04:17.179263 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.179237    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb" (OuterVolumeSpecName: "kube-api-access-qfqqb") pod "9df853ec-2f49-4695-9693-c358c0c1b7fc" (UID: "9df853ec-2f49-4695-9693-c358c0c1b7fc"). InnerVolumeSpecName "kube-api-access-qfqqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:04:17.276353 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276316    2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-cacerts\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276353 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276356    2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9df853ec-2f49-4695-9693-c358c0c1b7fc-local-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276374    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-ca-configmap\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276387    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276401    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-csr-dns-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276413    2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9df853ec-2f49-4695-9693-c358c0c1b7fc-istio-kubeconfig\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.276600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.276427    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfqqb\" (UniqueName: \"kubernetes.io/projected/9df853ec-2f49-4695-9693-c358c0c1b7fc-kube-api-access-qfqqb\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:04:17.809624 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.809027    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-9d52g"
Jul 07 00:04:17.876913 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.875583    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"
Jul 07 00:04:17.876913 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.875601    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" event={"ID":"9df853ec-2f49-4695-9693-c358c0c1b7fc","Type":"ContainerDied","Data":"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"}
Jul 07 00:04:17.876913 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.875706    2583 scope.go:117] "RemoveContainer" containerID="08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"
Jul 07 00:04:17.876913 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.875423    2583 generic.go:356] "Generic (PLEG): container finished" podID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerID="08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62" exitCode=0
Jul 07 00:04:17.876913 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.876326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5" event={"ID":"9df853ec-2f49-4695-9693-c358c0c1b7fc","Type":"ContainerDied","Data":"da0d5fafee93d6b685d4db2ddedb205f65ce3a60692b176729637fdea0f5f861"}
Jul 07 00:04:17.894376 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.894276    2583 scope.go:117] "RemoveContainer" containerID="08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"
Jul 07 00:04:17.894670 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:04:17.894651    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62\": container with ID starting with 08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62 not found: ID does not exist" containerID="08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"
Jul 07 00:04:17.894751 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.894685    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62"} err="failed to get container status \"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62\": rpc error: code = NotFound desc = could not find container \"08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62\": container with ID starting with 08e507d6d1eea5f5d6cde30b5211c652df44274dff457f2b1267df58a4d11d62 not found: ID does not exist"
Jul 07 00:04:17.917210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.917177    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:17.926254 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:17.926224    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-8wxk5"]
Jul 07 00:04:18.890816 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:18.890778    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm" event={"ID":"3e915ee8-6de5-4fd8-90cb-86e82fad7377","Type":"ContainerStarted","Data":"7139cde3730aa63de6211c1d2f85582498d8e031939862e823b5b40ba0d470cc"}
Jul 07 00:04:18.891275 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:18.890872    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:04:18.892470 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:18.892434    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5" event={"ID":"a5b59986-fc18-4fd7-8bd9-ec3803d58fa0","Type":"ContainerStarted","Data":"120c0de6f93d480d398fb30da2c46c2076d6d504f333d30450a6a55abb7c1ab4"}
Jul 07 00:04:18.892589 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:18.892507    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:18.918850 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:18.918418    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm" podStartSLOduration=2.409535339 podStartE2EDuration="4.918398213s" totalImagesPullingTime="2.508862874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:18.917489576 +0000 UTC m=+444.242492860" watchObservedRunningTime="2026-07-07 00:04:18.918398213 +0000 UTC m=+444.243401499"
Jul 07 00:04:19.132721 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.132677    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:04:19.132869 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.132761    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:04:19.132869 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.132791    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:04:19.159499 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.159474    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" path="/var/lib/kubelet/pods/9df853ec-2f49-4695-9693-c358c0c1b7fc/volumes"
Jul 07 00:04:19.224623 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.224568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.374685 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.374648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:04:19.826148 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.826118    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kqksn"
Jul 07 00:04:19.846676 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.846622    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5" podStartSLOduration=2.541337065 podStartE2EDuration="5.846604846s" totalImagesPullingTime="3.305267781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:18.959163678 +0000 UTC m=+444.284166962" watchObservedRunningTime="2026-07-07 00:04:19.846604846 +0000 UTC m=+445.171608130"
Jul 07 00:04:19.899929 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:19.899894    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s" event={"ID":"42796cd9-a85e-40ea-9d1a-4bd12addb356","Type":"ContainerStarted","Data":"c0cc36fb1dbea37f9dd4b218ddca690294610fdc8df61c228be28238c75babdd"}
Jul 07 00:04:20.449605 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:20.449566    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:20.454189 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:20.454163    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:20.491933 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:20.491877    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s" podStartSLOduration=2.149543262 podStartE2EDuration="5.491859571s" totalImagesPullingTime="3.342316309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:04:19.92319728 +0000 UTC m=+445.248200564" watchObservedRunningTime="2026-07-07 00:04:20.491859571 +0000 UTC m=+445.816862916"
Jul 07 00:04:20.903901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:20.903874    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:20.904815 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:20.904792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4khb4s"
Jul 07 00:04:49.902084 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:49.902053    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-2lbc5"
Jul 07 00:04:49.905035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:04:49.905011    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2vdcm"
Jul 07 00:05:03.248557 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.248513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:03.346529 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.346488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:03.382800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.382762    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:03.383117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.383104    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery"
Jul 07 00:05:03.383183 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.383119    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery"
Jul 07 00:05:03.383219 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.383193    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df853ec-2f49-4695-9693-c358c0c1b7fc" containerName="discovery"
Jul 07 00:05:03.391334 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.391278    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:03.394673 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.394253    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-7nhbs\""
Jul 07 00:05:03.394673 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.394359    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 07 00:05:03.394673 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.394516    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 07 00:05:03.395631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.395608    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:03.445606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.445568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:03.477025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.476985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp2wm\" (UniqueName: \"kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm\") pod \"kuadrant-operator-catalog-r2js9\" (UID: \"70ae504a-f191-4a1d-b83e-829064e87b97\") " pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:03.577743 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.577654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp2wm\" (UniqueName: \"kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm\") pod \"kuadrant-operator-catalog-r2js9\" (UID: \"70ae504a-f191-4a1d-b83e-829064e87b97\") " pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:03.585963 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.585937    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp2wm\" (UniqueName: \"kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm\") pod \"kuadrant-operator-catalog-r2js9\" (UID: \"70ae504a-f191-4a1d-b83e-829064e87b97\") " pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:03.702725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.702686    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:03.756589 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.756562    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:03.835728 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.835671    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:03.838122 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:03.838096    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ae504a_f191_4a1d_b83e_829064e87b97.slice/crio-ef47b326849f242e33319ec804effc4bbe193ccbb919b100db5746df139068b0 WatchSource:0}: Error finding container ef47b326849f242e33319ec804effc4bbe193ccbb919b100db5746df139068b0: Status 404 returned error can't find the container with id ef47b326849f242e33319ec804effc4bbe193ccbb919b100db5746df139068b0
Jul 07 00:05:03.960238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.960198    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-99gsc"]
Jul 07 00:05:03.963090 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.963074    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:03.973095 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.973072    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-99gsc"]
Jul 07 00:05:03.981406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:03.981372    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqt8n\" (UniqueName: \"kubernetes.io/projected/5f46530c-483c-4e73-92a6-c3922ab9227e-kube-api-access-bqt8n\") pod \"kuadrant-operator-catalog-99gsc\" (UID: \"5f46530c-483c-4e73-92a6-c3922ab9227e\") " pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:04.082566 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.082534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqt8n\" (UniqueName: \"kubernetes.io/projected/5f46530c-483c-4e73-92a6-c3922ab9227e-kube-api-access-bqt8n\") pod \"kuadrant-operator-catalog-99gsc\" (UID: \"5f46530c-483c-4e73-92a6-c3922ab9227e\") " pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:04.083793 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.083769    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" event={"ID":"70ae504a-f191-4a1d-b83e-829064e87b97","Type":"ContainerStarted","Data":"ef47b326849f242e33319ec804effc4bbe193ccbb919b100db5746df139068b0"}
Jul 07 00:05:04.094962 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.094894    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqt8n\" (UniqueName: \"kubernetes.io/projected/5f46530c-483c-4e73-92a6-c3922ab9227e-kube-api-access-bqt8n\") pod \"kuadrant-operator-catalog-99gsc\" (UID: \"5f46530c-483c-4e73-92a6-c3922ab9227e\") " pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:04.194852 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.194811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:04.272964 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.272929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:04.394074 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.394037    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:04.406935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:04.406906    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-99gsc"]
Jul 07 00:05:04.409160 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:04.409134    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f46530c_483c_4e73_92a6_c3922ab9227e.slice/crio-d44313f792efc34b40bea9b8624b0a4a19f932791712fcd896707767cac9a176 WatchSource:0}: Error finding container d44313f792efc34b40bea9b8624b0a4a19f932791712fcd896707767cac9a176: Status 404 returned error can't find the container with id d44313f792efc34b40bea9b8624b0a4a19f932791712fcd896707767cac9a176
Jul 07 00:05:05.091245 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:05.091198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-99gsc" event={"ID":"5f46530c-483c-4e73-92a6-c3922ab9227e","Type":"ContainerStarted","Data":"d44313f792efc34b40bea9b8624b0a4a19f932791712fcd896707767cac9a176"}
Jul 07 00:05:06.099808 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.099770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-99gsc" event={"ID":"5f46530c-483c-4e73-92a6-c3922ab9227e","Type":"ContainerStarted","Data":"b9f92a9413f73197cfef053d807a4ebe77e0341196898ce4a3a0eaff8d09cb2b"}
Jul 07 00:05:06.101405 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.101370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" event={"ID":"70ae504a-f191-4a1d-b83e-829064e87b97","Type":"ContainerStarted","Data":"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"}
Jul 07 00:05:06.101554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.101430    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" podUID="70ae504a-f191-4a1d-b83e-829064e87b97" containerName="registry-server" containerID="cri-o://082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4" gracePeriod=2
Jul 07 00:05:06.129979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.129928    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" podStartSLOduration=1.006580419 podStartE2EDuration="3.129907804s" totalImagesPullingTime="2.123327385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:06.129443696 +0000 UTC m=+491.454446983" watchObservedRunningTime="2026-07-07 00:05:06.129907804 +0000 UTC m=+491.454911088"
Jul 07 00:05:06.130147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.130010    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-99gsc" podStartSLOduration=1.5768134310000002 podStartE2EDuration="3.130004935s" totalImagesPullingTime="1.553191504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:06.115664784 +0000 UTC m=+491.440668069" watchObservedRunningTime="2026-07-07 00:05:06.130004935 +0000 UTC m=+491.455008222"
Jul 07 00:05:06.347760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.347728    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:06.401108 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.401070    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp2wm\" (UniqueName: \"kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm\") pod \"70ae504a-f191-4a1d-b83e-829064e87b97\" (UID: \"70ae504a-f191-4a1d-b83e-829064e87b97\") "
Jul 07 00:05:06.403276 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.403249    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm" (OuterVolumeSpecName: "kube-api-access-zp2wm") pod "70ae504a-f191-4a1d-b83e-829064e87b97" (UID: "70ae504a-f191-4a1d-b83e-829064e87b97"). InnerVolumeSpecName "kube-api-access-zp2wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:05:06.502528 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:06.502435    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp2wm\" (UniqueName: \"kubernetes.io/projected/70ae504a-f191-4a1d-b83e-829064e87b97-kube-api-access-zp2wm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:07.107182 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.107143    2583 generic.go:356] "Generic (PLEG): container finished" podID="70ae504a-f191-4a1d-b83e-829064e87b97" containerID="082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4" exitCode=0
Jul 07 00:05:07.107649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.107187    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r2js9"
Jul 07 00:05:07.107649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.107181    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" event={"ID":"70ae504a-f191-4a1d-b83e-829064e87b97","Type":"ContainerDied","Data":"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"}
Jul 07 00:05:07.107649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.107332    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r2js9" event={"ID":"70ae504a-f191-4a1d-b83e-829064e87b97","Type":"ContainerDied","Data":"ef47b326849f242e33319ec804effc4bbe193ccbb919b100db5746df139068b0"}
Jul 07 00:05:07.107649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.107366    2583 scope.go:117] "RemoveContainer" containerID="082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"
Jul 07 00:05:07.116690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.116673    2583 scope.go:117] "RemoveContainer" containerID="082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"
Jul 07 00:05:07.116904 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:05:07.116886    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4\": container with ID starting with 082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4 not found: ID does not exist" containerID="082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"
Jul 07 00:05:07.116954 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.116914    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4"} err="failed to get container status \"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4\": rpc error: code = NotFound desc = could not find container \"082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4\": container with ID starting with 082863b7f8796253fb2210f711f4a436d8ffe3185cac8dc530c268f0d1ab8ce4 not found: ID does not exist"
Jul 07 00:05:07.127286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.127264    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:07.132685 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.132658    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r2js9"]
Jul 07 00:05:07.157554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:07.157528    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70ae504a-f191-4a1d-b83e-829064e87b97" path="/var/lib/kubelet/pods/70ae504a-f191-4a1d-b83e-829064e87b97/volumes"
Jul 07 00:05:11.616316 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.616265    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79f496b6c-j897w"]
Jul 07 00:05:11.616684 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.616584    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70ae504a-f191-4a1d-b83e-829064e87b97" containerName="registry-server"
Jul 07 00:05:11.616684 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.616596    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="70ae504a-f191-4a1d-b83e-829064e87b97" containerName="registry-server"
Jul 07 00:05:11.616684 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.616655    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="70ae504a-f191-4a1d-b83e-829064e87b97" containerName="registry-server"
Jul 07 00:05:11.621592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.621571    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.633886 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.633860    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f496b6c-j897w"]
Jul 07 00:05:11.670458 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.670414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:11.730418 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.730393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:11.749578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-oauth-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749603    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqdl4\" (UniqueName: \"kubernetes.io/projected/99c40444-d40a-4890-bf47-9e2929fb28ca-kube-api-access-wqdl4\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-trusted-ca-bundle\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749656    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-console-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-oauth-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-service-ca\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.749816 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.749768    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850354 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850321    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqdl4\" (UniqueName: \"kubernetes.io/projected/99c40444-d40a-4890-bf47-9e2929fb28ca-kube-api-access-wqdl4\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850354 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-trusted-ca-bundle\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-console-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-oauth-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850427    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-service-ca\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850458    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.850532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.850513    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-oauth-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.851309 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.851260    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-oauth-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.851309 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.851260    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-service-ca\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.851498 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.851335    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-trusted-ca-bundle\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.851498 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.851350    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99c40444-d40a-4890-bf47-9e2929fb28ca-console-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.852986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.852964    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-serving-cert\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.853060 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.852994    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99c40444-d40a-4890-bf47-9e2929fb28ca-console-oauth-config\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.858866 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.858843    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqdl4\" (UniqueName: \"kubernetes.io/projected/99c40444-d40a-4890-bf47-9e2929fb28ca-kube-api-access-wqdl4\") pod \"console-79f496b6c-j897w\" (UID: \"99c40444-d40a-4890-bf47-9e2929fb28ca\") " pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:11.930897 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:11.930866    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:12.064804 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:12.064779    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79f496b6c-j897w"]
Jul 07 00:05:12.066991 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:12.066960    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c40444_d40a_4890_bf47_9e2929fb28ca.slice/crio-1303e711211e8a8e42f169eba5c6a06738fd31ce094b86614b993db022b477f1 WatchSource:0}: Error finding container 1303e711211e8a8e42f169eba5c6a06738fd31ce094b86614b993db022b477f1: Status 404 returned error can't find the container with id 1303e711211e8a8e42f169eba5c6a06738fd31ce094b86614b993db022b477f1
Jul 07 00:05:12.130794 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:12.130756    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f496b6c-j897w" event={"ID":"99c40444-d40a-4890-bf47-9e2929fb28ca","Type":"ContainerStarted","Data":"b4aa09948b933d879e1292809838a01359351052f6331c8cc3882799dd121d63"}
Jul 07 00:05:12.130930 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:12.130822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79f496b6c-j897w" event={"ID":"99c40444-d40a-4890-bf47-9e2929fb28ca","Type":"ContainerStarted","Data":"1303e711211e8a8e42f169eba5c6a06738fd31ce094b86614b993db022b477f1"}
Jul 07 00:05:12.152322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:12.152252    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79f496b6c-j897w" podStartSLOduration=1.152235654 podStartE2EDuration="1.152235654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:12.150504358 +0000 UTC m=+497.475507665" watchObservedRunningTime="2026-07-07 00:05:12.152235654 +0000 UTC m=+497.477238946"
Jul 07 00:05:14.273749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:14.273710    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:14.274185 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:14.273803    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:14.295753 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:14.295729    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:15.166582 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:15.166558    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-99gsc"
Jul 07 00:05:21.931632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:21.931591    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:21.932126 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:21.931664    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:21.936459 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:21.936434    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:22.176061 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.176033    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79f496b6c-j897w"
Jul 07 00:05:22.223223 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.223126    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:05:22.336830 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.336797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:22.437875 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.437840    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:22.543017 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.542936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:22.736549 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:22.736517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:31.662253 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:31.662197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:32.252096 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:32.252052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:33.281765 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:33.281715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:35.732766 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.732729    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"]
Jul 07 00:05:35.740382 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.740364    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:35.743280 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.743262    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-dd2fs\""
Jul 07 00:05:35.746543 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.746517    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"]
Jul 07 00:05:35.862383 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.862346    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn8t9\" (UniqueName: \"kubernetes.io/projected/065d5038-8b3b-4f7b-824e-5ddd409e6e4b-kube-api-access-vn8t9\") pod \"authorino-operator-75cc8d5cb7-9bjnd\" (UID: \"065d5038-8b3b-4f7b-824e-5ddd409e6e4b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:35.963459 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.963418    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn8t9\" (UniqueName: \"kubernetes.io/projected/065d5038-8b3b-4f7b-824e-5ddd409e6e4b-kube-api-access-vn8t9\") pod \"authorino-operator-75cc8d5cb7-9bjnd\" (UID: \"065d5038-8b3b-4f7b-824e-5ddd409e6e4b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:35.972586 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:35.972560    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn8t9\" (UniqueName: \"kubernetes.io/projected/065d5038-8b3b-4f7b-824e-5ddd409e6e4b-kube-api-access-vn8t9\") pod \"authorino-operator-75cc8d5cb7-9bjnd\" (UID: \"065d5038-8b3b-4f7b-824e-5ddd409e6e4b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:36.052080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:36.051998    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:36.203501 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:36.203477    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"]
Jul 07 00:05:36.206169 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:36.206137    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065d5038_8b3b_4f7b_824e_5ddd409e6e4b.slice/crio-88a2da4e8aba01f16bfcab18cc5ea74f2c4362b442055548046d898e8f28581c WatchSource:0}: Error finding container 88a2da4e8aba01f16bfcab18cc5ea74f2c4362b442055548046d898e8f28581c: Status 404 returned error can't find the container with id 88a2da4e8aba01f16bfcab18cc5ea74f2c4362b442055548046d898e8f28581c
Jul 07 00:05:36.235231 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:36.235206    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd" event={"ID":"065d5038-8b3b-4f7b-824e-5ddd409e6e4b","Type":"ContainerStarted","Data":"88a2da4e8aba01f16bfcab18cc5ea74f2c4362b442055548046d898e8f28581c"}
Jul 07 00:05:36.862032 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:36.861995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:38.718807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.718773    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"]
Jul 07 00:05:38.722241 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.722215    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:38.725487 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.725462    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rvrp2\""
Jul 07 00:05:38.725614 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.725462    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 07 00:05:38.735841 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.735816    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"]
Jul 07 00:05:38.898797 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.898754    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxnp2\" (UniqueName: \"kubernetes.io/projected/a98f2b02-4133-41a9-8fe9-d07782183190-kube-api-access-mxnp2\") pod \"dns-operator-controller-manager-864dd78c4c-df5h6\" (UID: \"a98f2b02-4133-41a9-8fe9-d07782183190\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:39.000042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:38.999954    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxnp2\" (UniqueName: \"kubernetes.io/projected/a98f2b02-4133-41a9-8fe9-d07782183190-kube-api-access-mxnp2\") pod \"dns-operator-controller-manager-864dd78c4c-df5h6\" (UID: \"a98f2b02-4133-41a9-8fe9-d07782183190\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:39.009811 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.009783    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxnp2\" (UniqueName: \"kubernetes.io/projected/a98f2b02-4133-41a9-8fe9-d07782183190-kube-api-access-mxnp2\") pod \"dns-operator-controller-manager-864dd78c4c-df5h6\" (UID: \"a98f2b02-4133-41a9-8fe9-d07782183190\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:39.032487 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.032459    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:39.171484 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.171459    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"]
Jul 07 00:05:39.173725 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:39.173697    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda98f2b02_4133_41a9_8fe9_d07782183190.slice/crio-c0c1e669178d8252f2b93694bf0943e77e355b7fe06f11a25336b2c5a9102dfa WatchSource:0}: Error finding container c0c1e669178d8252f2b93694bf0943e77e355b7fe06f11a25336b2c5a9102dfa: Status 404 returned error can't find the container with id c0c1e669178d8252f2b93694bf0943e77e355b7fe06f11a25336b2c5a9102dfa
Jul 07 00:05:39.250481 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.250389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6" event={"ID":"a98f2b02-4133-41a9-8fe9-d07782183190","Type":"ContainerStarted","Data":"c0c1e669178d8252f2b93694bf0943e77e355b7fe06f11a25336b2c5a9102dfa"}
Jul 07 00:05:39.251942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.251913    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd" event={"ID":"065d5038-8b3b-4f7b-824e-5ddd409e6e4b","Type":"ContainerStarted","Data":"1fd71e57b4b9aea2e5819afe38466650f5ac191ca78bdd1876a22cb5cb33a9fd"}
Jul 07 00:05:39.252060 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.252025    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:39.270631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.270584    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd" podStartSLOduration=1.999900072 podStartE2EDuration="4.27056939s" totalImagesPullingTime="2.270669318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:39.269678435 +0000 UTC m=+524.594681719" watchObservedRunningTime="2026-07-07 00:05:39.27056939 +0000 UTC m=+524.595572741"
Jul 07 00:05:39.449101 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.449066    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:39.833791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:39.833755    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:41.262705 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.262671    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6" event={"ID":"a98f2b02-4133-41a9-8fe9-d07782183190","Type":"ContainerStarted","Data":"8f29b9dae4247ec4e6053af9485e35ad9797e6dff96cb0a727a3e6e5abebc6f0"}
Jul 07 00:05:41.263183 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.262795    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:41.284577 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.284528    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6" podStartSLOduration=1.31778031 podStartE2EDuration="3.28451294s" totalImagesPullingTime="1.96673263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:41.282859633 +0000 UTC m=+526.607862916" watchObservedRunningTime="2026-07-07 00:05:41.28451294 +0000 UTC m=+526.609516225"
Jul 07 00:05:41.434193 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.434112    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"]
Jul 07 00:05:41.437410 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.437393    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:41.439815 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.439794    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-rgvrd\""
Jul 07 00:05:41.449331 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.449307    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"]
Jul 07 00:05:41.622088 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.622052    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trfqk\" (UniqueName: \"kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk\") pod \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" (UID: \"96ef186d-756f-48a6-b6f4-8bb20ed0fff4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:41.722958 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.722872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trfqk\" (UniqueName: \"kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk\") pod \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" (UID: \"96ef186d-756f-48a6-b6f4-8bb20ed0fff4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:41.736075 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.736044    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trfqk\" (UniqueName: \"kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk\") pod \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" (UID: \"96ef186d-756f-48a6-b6f4-8bb20ed0fff4\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:41.747835 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.747809    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:41.877894 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:41.877867    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"]
Jul 07 00:05:41.879176 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:41.879148    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ef186d_756f_48a6_b6f4_8bb20ed0fff4.slice/crio-9754d78fbd9f42aa45b0e635856f34110a01b9e0cca95c2f08c754c7418f369e WatchSource:0}: Error finding container 9754d78fbd9f42aa45b0e635856f34110a01b9e0cca95c2f08c754c7418f369e: Status 404 returned error can't find the container with id 9754d78fbd9f42aa45b0e635856f34110a01b9e0cca95c2f08c754c7418f369e
Jul 07 00:05:42.269102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:42.269067    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5" event={"ID":"96ef186d-756f-48a6-b6f4-8bb20ed0fff4","Type":"ContainerStarted","Data":"9754d78fbd9f42aa45b0e635856f34110a01b9e0cca95c2f08c754c7418f369e"}
Jul 07 00:05:42.325478 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:42.325442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:42.490395 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:42.490359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:44.280204 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:44.280172    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5" event={"ID":"96ef186d-756f-48a6-b6f4-8bb20ed0fff4","Type":"ContainerStarted","Data":"2b379b4428399ea22ef8063ffb313c8b59302d33fb9aa1024e528d85dad1f0a7"}
Jul 07 00:05:44.280622 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:44.280332    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:44.299803 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:44.299758    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5" podStartSLOduration=1.374302823 podStartE2EDuration="3.299742237s" totalImagesPullingTime="1.925439414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:44.297720759 +0000 UTC m=+529.622724057" watchObservedRunningTime="2026-07-07 00:05:44.299742237 +0000 UTC m=+529.624745521"
Jul 07 00:05:44.915822 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:44.915791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:46.146687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.146651    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"]
Jul 07 00:05:46.150056 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.150033    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.152760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.152735    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q4b5q\""
Jul 07 00:05:46.153328 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.153309    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.153421 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.153373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbl28\" (UniqueName: \"kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.165259 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.165239    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"]
Jul 07 00:05:46.254226 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.254188    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbl28\" (UniqueName: \"kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.254449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.254257    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.254709 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.254686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.263092 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.263059    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbl28\" (UniqueName: \"kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28\") pod \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.460594 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.460518    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:46.597364 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:46.597336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"]
Jul 07 00:05:46.600221 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:05:46.600190    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa7b7b7c_4bd2_4a4c_a62b_a3fc0a561f3a.slice/crio-efc1adbda33cf407f4d56b5670affc16488db9b7685f28d077c334906a54e669 WatchSource:0}: Error finding container efc1adbda33cf407f4d56b5670affc16488db9b7685f28d077c334906a54e669: Status 404 returned error can't find the container with id efc1adbda33cf407f4d56b5670affc16488db9b7685f28d077c334906a54e669
Jul 07 00:05:47.247549 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.247494    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-568b686dd4-r6rkh" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerName="console" containerID="cri-o://92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb" gracePeriod=15
Jul 07 00:05:47.294462 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.294428    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2" event={"ID":"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a","Type":"ContainerStarted","Data":"efc1adbda33cf407f4d56b5670affc16488db9b7685f28d077c334906a54e669"}
Jul 07 00:05:47.363955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.363926    2583 patch_prober.go:28] interesting pod/console-568b686dd4-r6rkh container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused" start-of-body=
Jul 07 00:05:47.364020 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.363987    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-568b686dd4-r6rkh" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerName="console" probeResult="failure" output="Get \"https://10.132.0.19:8443/health\": dial tcp 10.132.0.19:8443: connect: connection refused"
Jul 07 00:05:47.452626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.452589    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:47.502404 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.502343    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:05:47.542779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.542737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:47.567823 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567789    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.567823 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567825    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567890    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6c6jw\" (UniqueName: \"kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567919    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567941    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567967    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.567997    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert\") pod \"a8667b5e-9f88-43ef-bb7f-1553a9872186\" (UID: \"a8667b5e-9f88-43ef-bb7f-1553a9872186\") "
Jul 07 00:05:47.568444 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.568398    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca" (OuterVolumeSpecName: "service-ca") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:05:47.568580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.568478    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:05:47.568580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.568507    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:05:47.568580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.568539    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config" (OuterVolumeSpecName: "console-config") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:05:47.570051 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.570022    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:05:47.570164 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.570078    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:05:47.570164 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.570110    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw" (OuterVolumeSpecName: "kube-api-access-6c6jw") pod "a8667b5e-9f88-43ef-bb7f-1553a9872186" (UID: "a8667b5e-9f88-43ef-bb7f-1553a9872186"). InnerVolumeSpecName "kube-api-access-6c6jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:05:47.649322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.649256    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669484    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6c6jw\" (UniqueName: \"kubernetes.io/projected/a8667b5e-9f88-43ef-bb7f-1553a9872186-kube-api-access-6c6jw\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669515    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669527    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-trusted-ca-bundle\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669543    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669558    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-oauth-serving-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669570    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8667b5e-9f88-43ef-bb7f-1553a9872186-console-oauth-config\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.669572 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.669584    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8667b5e-9f88-43ef-bb7f-1553a9872186-service-ca\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:05:47.754097 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:47.754007    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:48.300879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.300838    2583 generic.go:356] "Generic (PLEG): container finished" podID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerID="92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb" exitCode=0
Jul 07 00:05:48.301274 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.300887    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-568b686dd4-r6rkh"
Jul 07 00:05:48.301274 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.300924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568b686dd4-r6rkh" event={"ID":"a8667b5e-9f88-43ef-bb7f-1553a9872186","Type":"ContainerDied","Data":"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"}
Jul 07 00:05:48.301274 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.300989    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-568b686dd4-r6rkh" event={"ID":"a8667b5e-9f88-43ef-bb7f-1553a9872186","Type":"ContainerDied","Data":"67e5dbdaeece0bb9849a757990fb42b7e83a7dd21638bac49aeec7d4b53e4498"}
Jul 07 00:05:48.301274 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.301014    2583 scope.go:117] "RemoveContainer" containerID="92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"
Jul 07 00:05:48.311393 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.311259    2583 scope.go:117] "RemoveContainer" containerID="92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"
Jul 07 00:05:48.311625 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:05:48.311578    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb\": container with ID starting with 92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb not found: ID does not exist" containerID="92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"
Jul 07 00:05:48.311701 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.311615    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb"} err="failed to get container status \"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb\": rpc error: code = NotFound desc = could not find container \"92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb\": container with ID starting with 92a8277587718bddb3aa654c74b2422c8ed6117e67a8f4dd257c6edfd551c4eb not found: ID does not exist"
Jul 07 00:05:48.328893 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.328861    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:05:48.331811 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:48.331785    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-568b686dd4-r6rkh"]
Jul 07 00:05:49.157784 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:49.157746    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" path="/var/lib/kubelet/pods/a8667b5e-9f88-43ef-bb7f-1553a9872186/volumes"
Jul 07 00:05:50.259001 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:50.258971    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9bjnd"
Jul 07 00:05:51.272887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:51.272851    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:51.373384 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:51.373331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:52.271403 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:52.271376    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-df5h6"
Jul 07 00:05:52.323954 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:52.323923    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2" event={"ID":"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a","Type":"ContainerStarted","Data":"12a41827baf36c43fec3fdb854b97d79acda5ada417294447579279974b3ae63"}
Jul 07 00:05:52.324349 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:52.324061    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:05:52.349338 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:52.349274    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2" podStartSLOduration=1.4248510699999999 podStartE2EDuration="6.349257346s" totalImagesPullingTime="4.924406276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:05:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:05:52.347498176 +0000 UTC m=+537.672501459" watchObservedRunningTime="2026-07-07 00:05:52.349257346 +0000 UTC m=+537.674260630"
Jul 07 00:05:54.263785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:54.263743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:54.801468 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:54.801432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:55.286417 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:55.286388    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:05:55.465282 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:55.465244    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:55.581284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:55.581254    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:58.911005 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:58.910970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:05:59.011613 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:05:59.011564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:03.331550 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:03.331516    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:06:05.133282 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.133251    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"]
Jul 07 00:06:05.133692 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.133468    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" containerName="manager" containerID="cri-o://12a41827baf36c43fec3fdb854b97d79acda5ada417294447579279974b3ae63" gracePeriod=2
Jul 07 00:06:05.150037 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.150005    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"]
Jul 07 00:06:05.155330 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:05.155279    2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\": pods \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:06:05.159719 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:05.159686    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-lv4k2\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:06:05.166329 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.166303    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:05.166846 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.166828    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerName="console"
Jul 07 00:06:05.166899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.166850    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerName="console"
Jul 07 00:06:05.166899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.166872    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" containerName="manager"
Jul 07 00:06:05.166899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.166881    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" containerName="manager"
Jul 07 00:06:05.167023 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.167010    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" containerName="manager"
Jul 07 00:06:05.167058 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.167031    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8667b5e-9f88-43ef-bb7f-1553a9872186" containerName="console"
Jul 07 00:06:05.170626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.170598    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"]
Jul 07 00:06:05.170743 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.170722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.171036 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.171005    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerName="manager" containerID="cri-o://2b379b4428399ea22ef8063ffb313c8b59302d33fb9aa1024e528d85dad1f0a7" gracePeriod=2
Jul 07 00:06:05.184259 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.184231    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"]
Jul 07 00:06:05.188589 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.188562    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:05.200242 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.200211    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"]
Jul 07 00:06:05.204072 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.201614    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerName="manager"
Jul 07 00:06:05.204072 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.201636    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerName="manager"
Jul 07 00:06:05.204072 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.201830    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerName="manager"
Jul 07 00:06:05.206959 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.206935    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:05.213869 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.213813    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"]
Jul 07 00:06:05.227213 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:05.227179    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:05.228601 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.228579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q67xp\" (UniqueName: \"kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.228705 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.228631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.329981 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.329941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4s28\" (UniqueName: \"kubernetes.io/projected/093daf68-6461-4c4e-acad-f643ee39b22d-kube-api-access-s4s28\") pod \"limitador-operator-controller-manager-859d7b78bd-q92m4\" (UID: \"093daf68-6461-4c4e-acad-f643ee39b22d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:05.330159 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.330015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q67xp\" (UniqueName: \"kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.330159 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.330069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.330472 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.330453    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.345103 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.345074    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q67xp\" (UniqueName: \"kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8sc7p\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.387798 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.387709    2583 generic.go:356] "Generic (PLEG): container finished" podID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" containerID="12a41827baf36c43fec3fdb854b97d79acda5ada417294447579279974b3ae63" exitCode=0
Jul 07 00:06:05.389530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.389504    2583 generic.go:356] "Generic (PLEG): container finished" podID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerID="2b379b4428399ea22ef8063ffb313c8b59302d33fb9aa1024e528d85dad1f0a7" exitCode=0
Jul 07 00:06:05.407989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.407971    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:06:05.410625 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:05.410597    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:05.410893 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.410879    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:05.412942 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:05.412919    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:05.431392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.431368    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4s28\" (UniqueName: \"kubernetes.io/projected/093daf68-6461-4c4e-acad-f643ee39b22d-kube-api-access-s4s28\") pod \"limitador-operator-controller-manager-859d7b78bd-q92m4\" (UID: \"093daf68-6461-4c4e-acad-f643ee39b22d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:05.445163 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.445141    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4s28\" (UniqueName: \"kubernetes.io/projected/093daf68-6461-4c4e-acad-f643ee39b22d-kube-api-access-s4s28\") pod \"limitador-operator-controller-manager-859d7b78bd-q92m4\" (UID: \"093daf68-6461-4c4e-acad-f643ee39b22d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:05.531947 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.531911    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trfqk\" (UniqueName: \"kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk\") pod \"96ef186d-756f-48a6-b6f4-8bb20ed0fff4\" (UID: \"96ef186d-756f-48a6-b6f4-8bb20ed0fff4\") "
Jul 07 00:06:05.532143 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.532004    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume\") pod \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") "
Jul 07 00:06:05.532143 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.532025    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbl28\" (UniqueName: \"kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28\") pod \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\" (UID: \"aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a\") "
Jul 07 00:06:05.532521 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.532410    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" (UID: "aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:06:05.534168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.534145    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk" (OuterVolumeSpecName: "kube-api-access-trfqk") pod "96ef186d-756f-48a6-b6f4-8bb20ed0fff4" (UID: "96ef186d-756f-48a6-b6f4-8bb20ed0fff4"). InnerVolumeSpecName "kube-api-access-trfqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:06:05.534269 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.534213    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28" (OuterVolumeSpecName: "kube-api-access-wbl28") pod "aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" (UID: "aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a"). InnerVolumeSpecName "kube-api-access-wbl28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:06:05.561033 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.561007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:05.566049 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.566028    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:05.633710 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.633500    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-extensions-socket-volume\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:06:05.633710 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.633538    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbl28\" (UniqueName: \"kubernetes.io/projected/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a-kube-api-access-wbl28\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:06:05.633710 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.633554    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trfqk\" (UniqueName: \"kubernetes.io/projected/96ef186d-756f-48a6-b6f4-8bb20ed0fff4-kube-api-access-trfqk\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:06:05.707806 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.707777    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:05.710620 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:05.710592    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7858366c_aa19_49ab_b43d_b0945dd362da.slice/crio-8df1094adb24b651624cd1dd4afc83f65292b5068a434f8b0b351d3a6b05c943 WatchSource:0}: Error finding container 8df1094adb24b651624cd1dd4afc83f65292b5068a434f8b0b351d3a6b05c943: Status 404 returned error can't find the container with id 8df1094adb24b651624cd1dd4afc83f65292b5068a434f8b0b351d3a6b05c943
Jul 07 00:06:05.728004 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:05.727984    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"]
Jul 07 00:06:05.730083 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:05.730022    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093daf68_6461_4c4e_acad_f643ee39b22d.slice/crio-9767e58203563746f9505133e4b935e3a4101fdc77b542cdd1bbd10ff18dcd3c WatchSource:0}: Error finding container 9767e58203563746f9505133e4b935e3a4101fdc77b542cdd1bbd10ff18dcd3c: Status 404 returned error can't find the container with id 9767e58203563746f9505133e4b935e3a4101fdc77b542cdd1bbd10ff18dcd3c
Jul 07 00:06:06.286413 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.286362    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.35:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:06:06.395603 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.395568    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" event={"ID":"7858366c-aa19-49ab-b43d-b0945dd362da","Type":"ContainerStarted","Data":"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"}
Jul 07 00:06:06.395781 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.395626    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" event={"ID":"7858366c-aa19-49ab-b43d-b0945dd362da","Type":"ContainerStarted","Data":"8df1094adb24b651624cd1dd4afc83f65292b5068a434f8b0b351d3a6b05c943"}
Jul 07 00:06:06.395781 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.395656    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:06.396829 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.396809    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:06.396998 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.396843    2583 scope.go:117] "RemoveContainer" containerID="2b379b4428399ea22ef8063ffb313c8b59302d33fb9aa1024e528d85dad1f0a7"
Jul 07 00:06:06.398125 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:06.398058    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:06.398194 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.398129    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-lv4k2"
Jul 07 00:06:06.399800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.399780    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4" event={"ID":"093daf68-6461-4c4e-acad-f643ee39b22d","Type":"ContainerStarted","Data":"2ecd62247cda28c6e9221e0051054b55a5bcd560a133a3742a5657ee8095f9a5"}
Jul 07 00:06:06.399896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.399819    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4" event={"ID":"093daf68-6461-4c4e-acad-f643ee39b22d","Type":"ContainerStarted","Data":"9767e58203563746f9505133e4b935e3a4101fdc77b542cdd1bbd10ff18dcd3c"}
Jul 07 00:06:06.399943 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.399919    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:06.406407 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.406367    2583 scope.go:117] "RemoveContainer" containerID="12a41827baf36c43fec3fdb854b97d79acda5ada417294447579279974b3ae63"
Jul 07 00:06:06.426741 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.426701    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" podStartSLOduration=1.426686302 podStartE2EDuration="1.426686302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:06.424938648 +0000 UTC m=+551.749941932" watchObservedRunningTime="2026-07-07 00:06:06.426686302 +0000 UTC m=+551.751689585"
Jul 07 00:06:06.427411 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:06.427379    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4tdz5\" is forbidden: User \"system:node:ip-10-0-142-92.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-92.ec2.internal' and this object" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4tdz5"
Jul 07 00:06:06.606347 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.606308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:06.706792 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.706700    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:06.806985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:06.806948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:07.096760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:07.096670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:07.157957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:07.157927    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ef186d-756f-48a6-b6f4-8bb20ed0fff4" path="/var/lib/kubelet/pods/96ef186d-756f-48a6-b6f4-8bb20ed0fff4/volumes"
Jul 07 00:06:07.158273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:07.158260    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a" path="/var/lib/kubelet/pods/aa7b7b7c-4bd2-4a4c-a62b-a3fc0a561f3a/volumes"
Jul 07 00:06:07.199643 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:07.199610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:09.696513 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:09.696474    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:09.757030 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:09.756996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:14.904597 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:14.904564    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:15.048938 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:15.048902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:17.408984 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:17.408952    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:17.409401 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:17.409031    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4"
Jul 07 00:06:17.430247 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:17.430196    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q92m4" podStartSLOduration=12.430180472 podStartE2EDuration="12.430180472s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:06.454349586 +0000 UTC m=+551.779352871" watchObservedRunningTime="2026-07-07 00:06:17.430180472 +0000 UTC m=+562.755183756"
Jul 07 00:06:18.560727 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:18.560688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:18.630559 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:18.630520    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:20.884057 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:20.884013    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:20.884552 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:20.884232    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" podUID="7858366c-aa19-49ab-b43d-b0945dd362da" containerName="manager" containerID="cri-o://10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b" gracePeriod=10
Jul 07 00:06:21.111382 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.111345    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:06:21.115518 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.115493    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.129503 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.129475    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:06:21.148100 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.148031    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:21.273654 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.273623    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q67xp\" (UniqueName: \"kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp\") pod \"7858366c-aa19-49ab-b43d-b0945dd362da\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") "
Jul 07 00:06:21.273848 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.273719    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume\") pod \"7858366c-aa19-49ab-b43d-b0945dd362da\" (UID: \"7858366c-aa19-49ab-b43d-b0945dd362da\") "
Jul 07 00:06:21.273944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.273923    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9mph\" (UniqueName: \"kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.274008 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.273967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.274153 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.274130    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7858366c-aa19-49ab-b43d-b0945dd362da" (UID: "7858366c-aa19-49ab-b43d-b0945dd362da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:06:21.275738 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.275718    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp" (OuterVolumeSpecName: "kube-api-access-q67xp") pod "7858366c-aa19-49ab-b43d-b0945dd362da" (UID: "7858366c-aa19-49ab-b43d-b0945dd362da"). InnerVolumeSpecName "kube-api-access-q67xp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:06:21.375022 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.374986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9mph\" (UniqueName: \"kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.375206 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.375034    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.375206 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.375093    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q67xp\" (UniqueName: \"kubernetes.io/projected/7858366c-aa19-49ab-b43d-b0945dd362da-kube-api-access-q67xp\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:06:21.375206 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.375103    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7858366c-aa19-49ab-b43d-b0945dd362da-extensions-socket-volume\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:06:21.375448 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.375432    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.383589 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.383568    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9mph\" (UniqueName: \"kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph\") pod \"kuadrant-operator-controller-manager-86878c769f-f4znx\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.425736 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.425639    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:21.472662 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.472621    2583 generic.go:356] "Generic (PLEG): container finished" podID="7858366c-aa19-49ab-b43d-b0945dd362da" containerID="10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b" exitCode=0
Jul 07 00:06:21.472662 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.472658    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"
Jul 07 00:06:21.473125 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.472701    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" event={"ID":"7858366c-aa19-49ab-b43d-b0945dd362da","Type":"ContainerDied","Data":"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"}
Jul 07 00:06:21.473125 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.472766    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p" event={"ID":"7858366c-aa19-49ab-b43d-b0945dd362da","Type":"ContainerDied","Data":"8df1094adb24b651624cd1dd4afc83f65292b5068a434f8b0b351d3a6b05c943"}
Jul 07 00:06:21.473125 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.472797    2583 scope.go:117] "RemoveContainer" containerID="10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"
Jul 07 00:06:21.485758 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.485712    2583 scope.go:117] "RemoveContainer" containerID="10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"
Jul 07 00:06:21.486075 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:06:21.486049    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b\": container with ID starting with 10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b not found: ID does not exist" containerID="10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"
Jul 07 00:06:21.486172 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.486087    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b"} err="failed to get container status \"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b\": rpc error: code = NotFound desc = could not find container \"10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b\": container with ID starting with 10a1bdc75a73dbfef30f4d2117da4d78bb62f0abab2a8b7667e6ae5c4b12de7b not found: ID does not exist"
Jul 07 00:06:21.513224 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.513194    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:21.520800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.520777    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8sc7p"]
Jul 07 00:06:21.566661 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:21.566636    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:06:21.568872 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:21.568850    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0358cb66_2534_483e_a56e_e099847da9ce.slice/crio-7cc40475f95e7b48a34f81067849415774ca806f9c49f5f278878807e3adf763 WatchSource:0}: Error finding container 7cc40475f95e7b48a34f81067849415774ca806f9c49f5f278878807e3adf763: Status 404 returned error can't find the container with id 7cc40475f95e7b48a34f81067849415774ca806f9c49f5f278878807e3adf763
Jul 07 00:06:22.478882 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:22.478845    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" event={"ID":"0358cb66-2534-483e-a56e-e099847da9ce","Type":"ContainerStarted","Data":"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"}
Jul 07 00:06:22.479369 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:22.478902    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" event={"ID":"0358cb66-2534-483e-a56e-e099847da9ce","Type":"ContainerStarted","Data":"7cc40475f95e7b48a34f81067849415774ca806f9c49f5f278878807e3adf763"}
Jul 07 00:06:22.479369 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:22.478927    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:22.500989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:22.500924    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" podStartSLOduration=1.500905404 podStartE2EDuration="1.500905404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:22.499719811 +0000 UTC m=+567.824723097" watchObservedRunningTime="2026-07-07 00:06:22.500905404 +0000 UTC m=+567.825908690"
Jul 07 00:06:23.158496 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:23.158466    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7858366c-aa19-49ab-b43d-b0945dd362da" path="/var/lib/kubelet/pods/7858366c-aa19-49ab-b43d-b0945dd362da/volumes"
Jul 07 00:06:32.938060 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:32.938018    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:33.486391 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:33.486353    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:06:37.276334 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.276285    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"]
Jul 07 00:06:37.276721 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.276656    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7858366c-aa19-49ab-b43d-b0945dd362da" containerName="manager"
Jul 07 00:06:37.276721 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.276667    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7858366c-aa19-49ab-b43d-b0945dd362da" containerName="manager"
Jul 07 00:06:37.276721 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.276721    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7858366c-aa19-49ab-b43d-b0945dd362da" containerName="manager"
Jul 07 00:06:37.280802 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.280785    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.283408 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.283380    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rwdkm\""
Jul 07 00:06:37.292702 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.292678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"]
Jul 07 00:06:37.305904 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.305875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:37.315864 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.315838    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgdbq\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-kube-api-access-lgdbq\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.315989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.315887    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.315989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.315948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.315989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.315978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.316136 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.316078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.316136 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.316126    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.316208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.316162    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.316208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.316193    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.316324 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.316273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.417581 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417547    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.417581 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417587    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417608    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417681    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417727    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgdbq\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-kube-api-access-lgdbq\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417799    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417824    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.417979    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418430 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.418128    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418430 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.418326    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418502 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.418426    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.418630 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.418612    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.420033 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.420010    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.420186 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.420169    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.426122 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.426100    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.426224 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.426148    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgdbq\" (UniqueName: \"kubernetes.io/projected/f826f9ad-2195-4107-b8e4-3d7ecee2ff27-kube-api-access-lgdbq\") pod \"maas-default-gateway-openshift-default-56654dcf7-trhm5\" (UID: \"f826f9ad-2195-4107-b8e4-3d7ecee2ff27\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.525905 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.525861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:37.592613 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.592513    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:37.732639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.732611    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"]
Jul 07 00:06:37.734927 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:37.734883    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf826f9ad_2195_4107_b8e4_3d7ecee2ff27.slice/crio-a578a69b97b2f3c464cf5bc9bf18d866e121532f5b6c239d7aff5244b552e007 WatchSource:0}: Error finding container a578a69b97b2f3c464cf5bc9bf18d866e121532f5b6c239d7aff5244b552e007: Status 404 returned error can't find the container with id a578a69b97b2f3c464cf5bc9bf18d866e121532f5b6c239d7aff5244b552e007
Jul 07 00:06:37.736937 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.736902    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:06:37.737039 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.736968    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:06:37.737039 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:37.737016    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:06:38.554668 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:38.554632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" event={"ID":"f826f9ad-2195-4107-b8e4-3d7ecee2ff27","Type":"ContainerStarted","Data":"a657302ff4146edca6ca9536ac41b0c9446fea1e7257adbe25043a23d0b0fde5"}
Jul 07 00:06:38.555068 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:38.554690    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" event={"ID":"f826f9ad-2195-4107-b8e4-3d7ecee2ff27","Type":"ContainerStarted","Data":"a578a69b97b2f3c464cf5bc9bf18d866e121532f5b6c239d7aff5244b552e007"}
Jul 07 00:06:38.577277 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:38.577223    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podStartSLOduration=1.577206797 podStartE2EDuration="1.577206797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:38.574576368 +0000 UTC m=+583.899579652" watchObservedRunningTime="2026-07-07 00:06:38.577206797 +0000 UTC m=+583.902210081"
Jul 07 00:06:38.592910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:38.592876    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:38.598342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:38.598314    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:39.562190 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:39.562152    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:39.565884 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:39.565858    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:06:52.795091 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:52.795048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:52.915950 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:52.915913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:52.967139 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:52.967098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:53.607826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.607790    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:06:53.611406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.611383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:06:53.613898 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.613872    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hw4v9\""
Jul 07 00:06:53.617906 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.617885    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:06:53.646807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.646777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:53.727629 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.727594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:06:53.750855 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.750821    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:06:53.754469 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.754445    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:06:53.761536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.761446    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:06:53.761536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.761498    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68bsk\" (UniqueName: \"kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk\") pod \"authorino-58cd955675-mzl2c\" (UID: \"45966a55-b99d-4fbd-8b1d-e2ac3b889443\") " pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:06:53.862531 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.862446    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svr6c\" (UniqueName: \"kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c\") pod \"authorino-75d556bd77-49sr6\" (UID: \"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf\") " pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:06:53.862531 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.862492    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68bsk\" (UniqueName: \"kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk\") pod \"authorino-58cd955675-mzl2c\" (UID: \"45966a55-b99d-4fbd-8b1d-e2ac3b889443\") " pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:06:53.870901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.870878    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68bsk\" (UniqueName: \"kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk\") pod \"authorino-58cd955675-mzl2c\" (UID: \"45966a55-b99d-4fbd-8b1d-e2ac3b889443\") " pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:06:53.922474 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.922443    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:06:53.963797 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.963765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svr6c\" (UniqueName: \"kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c\") pod \"authorino-75d556bd77-49sr6\" (UID: \"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf\") " pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:06:53.977232 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:53.977170    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svr6c\" (UniqueName: \"kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c\") pod \"authorino-75d556bd77-49sr6\" (UID: \"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf\") " pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:06:54.064346 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:54.064316    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:06:54.064440 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:54.064415    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:06:54.066107 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:54.066076    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45966a55_b99d_4fbd_8b1d_e2ac3b889443.slice/crio-f0e178b2de0a0faadee84381e6f37452791255fcc365954986288e61951e1fa0 WatchSource:0}: Error finding container f0e178b2de0a0faadee84381e6f37452791255fcc365954986288e61951e1fa0: Status 404 returned error can't find the container with id f0e178b2de0a0faadee84381e6f37452791255fcc365954986288e61951e1fa0
Jul 07 00:06:54.204410 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:54.204385    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:06:54.206693 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:06:54.206665    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeac0f36e_60ce_4765_bc47_4a52ea3c8ebf.slice/crio-7c1e3d008ba8b4c5368cf3b4835b77e562f3015bacebc14001de9bde5f0d3565 WatchSource:0}: Error finding container 7c1e3d008ba8b4c5368cf3b4835b77e562f3015bacebc14001de9bde5f0d3565: Status 404 returned error can't find the container with id 7c1e3d008ba8b4c5368cf3b4835b77e562f3015bacebc14001de9bde5f0d3565
Jul 07 00:06:54.631366 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:54.631325    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-49sr6" event={"ID":"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf","Type":"ContainerStarted","Data":"7c1e3d008ba8b4c5368cf3b4835b77e562f3015bacebc14001de9bde5f0d3565"}
Jul 07 00:06:54.632446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:54.632421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mzl2c" event={"ID":"45966a55-b99d-4fbd-8b1d-e2ac3b889443","Type":"ContainerStarted","Data":"f0e178b2de0a0faadee84381e6f37452791255fcc365954986288e61951e1fa0"}
Jul 07 00:06:55.121381 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:55.121355    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:06:55.122341 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:55.122315    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:06:58.663715 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:58.663679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-49sr6" event={"ID":"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf","Type":"ContainerStarted","Data":"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"}
Jul 07 00:06:58.665193 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:58.665166    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mzl2c" event={"ID":"45966a55-b99d-4fbd-8b1d-e2ac3b889443","Type":"ContainerStarted","Data":"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"}
Jul 07 00:06:58.679342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:58.679282    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-49sr6" podStartSLOduration=2.096157618 podStartE2EDuration="5.679267463s" totalImagesPullingTime="3.583109845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:58.677733715 +0000 UTC m=+604.002736999" watchObservedRunningTime="2026-07-07 00:06:58.679267463 +0000 UTC m=+604.004270747"
Jul 07 00:06:58.691612 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:58.691568    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mzl2c" podStartSLOduration=1.9568190699999999 podStartE2EDuration="5.691550023s" totalImagesPullingTime="3.734730953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:06:58.691227793 +0000 UTC m=+604.016231078" watchObservedRunningTime="2026-07-07 00:06:58.691550023 +0000 UTC m=+604.016553309"
Jul 07 00:06:58.720320 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:06:58.720265    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:07:00.674475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:00.674432    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mzl2c" podUID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" containerName="authorino" containerID="cri-o://d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58" gracePeriod=30
Jul 07 00:07:00.926667 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:00.926610    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:07:01.031048 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.031012    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68bsk\" (UniqueName: \"kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk\") pod \"45966a55-b99d-4fbd-8b1d-e2ac3b889443\" (UID: \"45966a55-b99d-4fbd-8b1d-e2ac3b889443\") "
Jul 07 00:07:01.033055 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.033029    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk" (OuterVolumeSpecName: "kube-api-access-68bsk") pod "45966a55-b99d-4fbd-8b1d-e2ac3b889443" (UID: "45966a55-b99d-4fbd-8b1d-e2ac3b889443"). InnerVolumeSpecName "kube-api-access-68bsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:07:01.132209 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.132167    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68bsk\" (UniqueName: \"kubernetes.io/projected/45966a55-b99d-4fbd-8b1d-e2ac3b889443-kube-api-access-68bsk\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:07:01.680358 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.680320    2583 generic.go:356] "Generic (PLEG): container finished" podID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" containerID="d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58" exitCode=0
Jul 07 00:07:01.680768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.680418    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mzl2c"
Jul 07 00:07:01.680768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.680462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mzl2c" event={"ID":"45966a55-b99d-4fbd-8b1d-e2ac3b889443","Type":"ContainerDied","Data":"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"}
Jul 07 00:07:01.680768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.680505    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mzl2c" event={"ID":"45966a55-b99d-4fbd-8b1d-e2ac3b889443","Type":"ContainerDied","Data":"f0e178b2de0a0faadee84381e6f37452791255fcc365954986288e61951e1fa0"}
Jul 07 00:07:01.680768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.680523    2583 scope.go:117] "RemoveContainer" containerID="d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"
Jul 07 00:07:01.689121 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.689105    2583 scope.go:117] "RemoveContainer" containerID="d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"
Jul 07 00:07:01.689376 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:07:01.689357    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58\": container with ID starting with d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58 not found: ID does not exist" containerID="d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"
Jul 07 00:07:01.689439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.689383    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58"} err="failed to get container status \"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58\": rpc error: code = NotFound desc = could not find container \"d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58\": container with ID starting with d5ae7a4151cdca7802fd5699eb507a3cbd48c19e1c9046057b88473d08a91a58 not found: ID does not exist"
Jul 07 00:07:01.697176 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.697154    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:07:01.699335 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:01.699317    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mzl2c"]
Jul 07 00:07:03.158262 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:03.158229    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" path="/var/lib/kubelet/pods/45966a55-b99d-4fbd-8b1d-e2ac3b889443/volumes"
Jul 07 00:07:07.034209 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:07.034172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:07.135237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:07.135150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:07.872356 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:07.872320    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:11.178536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:11.178491    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:11.277240 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:11.277209    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:11.710676 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:11.710630    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:21.536833 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:21.536752    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:22.969031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:22.968981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:23.519952 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:23.519922    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:23.621197 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:23.621125    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:24.039085 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.039044    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:07:24.039600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.039577    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" containerName="authorino"
Jul 07 00:07:24.039680 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.039604    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" containerName="authorino"
Jul 07 00:07:24.039763 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.039748    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="45966a55-b99d-4fbd-8b1d-e2ac3b889443" containerName="authorino"
Jul 07 00:07:24.042914 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.042890    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.045357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.045339    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 07 00:07:24.051957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.051934    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:07:24.121076 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.121045    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zn24\" (UniqueName: \"kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.121269 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.121114    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.222245 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.222205    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zn24\" (UniqueName: \"kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.222483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.222327    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.224835 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.224814    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.230814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.230792    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zn24\" (UniqueName: \"kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24\") pod \"authorino-84bb5bc755-5w7vk\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") " pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.353149 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.353063    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:07:24.490898 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.490864    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:07:24.493459 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:07:24.493433    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode17fc9bd_577d_45a3_9b4a_1b5b4ce3587b.slice/crio-2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c WatchSource:0}: Error finding container 2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c: Status 404 returned error can't find the container with id 2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c
Jul 07 00:07:24.495254 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.495237    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:07:24.788157 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:24.788118    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" event={"ID":"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b","Type":"ContainerStarted","Data":"2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c"}
Jul 07 00:07:25.793827 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:25.793790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" event={"ID":"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b","Type":"ContainerStarted","Data":"fefb443041f4b4363a27b66f4058e48406ff4f38b52606e538ea95875c5bb33d"}
Jul 07 00:07:25.809490 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:25.809437    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" podStartSLOduration=1.455254915 podStartE2EDuration="1.809420299s" totalImagesPullingTime="354.165384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:07:25.80885824 +0000 UTC m=+631.133861525" watchObservedRunningTime="2026-07-07 00:07:25.809420299 +0000 UTC m=+631.134423580"
Jul 07 00:07:25.834638 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:25.834603    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:07:25.835262 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:25.834828    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-49sr6" podUID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" containerName="authorino" containerID="cri-o://250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b" gracePeriod=30
Jul 07 00:07:26.096623 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.096602    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:07:26.139349 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.139316    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svr6c\" (UniqueName: \"kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c\") pod \"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf\" (UID: \"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf\") "
Jul 07 00:07:26.141466 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.141439    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c" (OuterVolumeSpecName: "kube-api-access-svr6c") pod "eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" (UID: "eac0f36e-60ce-4765-bc47-4a52ea3c8ebf"). InnerVolumeSpecName "kube-api-access-svr6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:07:26.240122 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.240089    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svr6c\" (UniqueName: \"kubernetes.io/projected/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf-kube-api-access-svr6c\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:07:26.799889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.799849    2583 generic.go:356] "Generic (PLEG): container finished" podID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" containerID="250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b" exitCode=0
Jul 07 00:07:26.800373 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.799900    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-49sr6"
Jul 07 00:07:26.800373 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.799899    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-49sr6" event={"ID":"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf","Type":"ContainerDied","Data":"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"}
Jul 07 00:07:26.800373 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.799942    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-49sr6" event={"ID":"eac0f36e-60ce-4765-bc47-4a52ea3c8ebf","Type":"ContainerDied","Data":"7c1e3d008ba8b4c5368cf3b4835b77e562f3015bacebc14001de9bde5f0d3565"}
Jul 07 00:07:26.800373 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.799960    2583 scope.go:117] "RemoveContainer" containerID="250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"
Jul 07 00:07:26.809066 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.809050    2583 scope.go:117] "RemoveContainer" containerID="250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"
Jul 07 00:07:26.809376 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:07:26.809360    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b\": container with ID starting with 250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b not found: ID does not exist" containerID="250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"
Jul 07 00:07:26.809446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.809388    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b"} err="failed to get container status \"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b\": rpc error: code = NotFound desc = could not find container \"250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b\": container with ID starting with 250d39857984aaa9053ad0ed2cf62f99ae968af18f21cfbe80d7d27a4e90a75b not found: ID does not exist"
Jul 07 00:07:26.821029 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.821006    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:07:26.823015 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:26.822994    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-49sr6"]
Jul 07 00:07:27.158877 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:27.158832    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" path="/var/lib/kubelet/pods/eac0f36e-60ce-4765-bc47-4a52ea3c8ebf/volumes"
Jul 07 00:07:30.305364 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:30.305323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:30.400558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:30.400521    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:30.472077 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:30.472044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:33.215424 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:33.215379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:33.333958 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:33.333927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:07:55.624662 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:07:55.624622    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:01.484574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:01.484534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:01.955846 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:01.955813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:02.393779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:02.393740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:02.503213 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:02.503159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:02.716115 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:02.716025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:09.652430 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:09.652390    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-trhm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:60146->10.132.0.40:15021: read: connection reset by peer" start-of-body=
Jul 07 00:08:09.652860 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:09.652476    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podUID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": read tcp 10.132.0.2:60146->10.132.0.40:15021: read: connection reset by peer"
Jul 07 00:08:10.009311 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.009203    2583 generic.go:356] "Generic (PLEG): container finished" podID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerID="a657302ff4146edca6ca9536ac41b0c9446fea1e7257adbe25043a23d0b0fde5" exitCode=0
Jul 07 00:08:10.009311 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.009280    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" event={"ID":"f826f9ad-2195-4107-b8e4-3d7ecee2ff27","Type":"ContainerDied","Data":"a657302ff4146edca6ca9536ac41b0c9446fea1e7257adbe25043a23d0b0fde5"}
Jul 07 00:08:10.009740 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.009721    2583 scope.go:117] "RemoveContainer" containerID="a657302ff4146edca6ca9536ac41b0c9446fea1e7257adbe25043a23d0b0fde5"
Jul 07 00:08:10.010697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.010669    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:08:10.010774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.010760    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:08:10.010827 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:10.010814    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:08:11.016375 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:11.016338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" event={"ID":"f826f9ad-2195-4107-b8e4-3d7ecee2ff27","Type":"ContainerStarted","Data":"c6c20de6e60c8028e126fdef697d1f0b65acbcc565c61caa6bb4d07889bee919"}
Jul 07 00:08:11.592713 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:11.592674    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:08:11.592935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:11.592911    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-trhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 00:08:11.593019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:11.592971    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podUID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 00:08:12.593618 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:12.593582    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-trhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 00:08:12.594006 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:12.593644    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podUID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 00:08:13.593642 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:13.593604    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-trhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 00:08:13.594055 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:13.593665    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podUID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 00:08:14.506484 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:14.506422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:14.593973 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:14.593932    2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-trhm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body=
Jul 07 00:08:14.594405 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:14.594008    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5" podUID="f826f9ad-2195-4107-b8e4-3d7ecee2ff27" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused"
Jul 07 00:08:14.599030 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:14.599000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:15.597157 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:15.597127    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:08:15.597592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:15.597418    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:08:15.598134 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:15.598117    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-trhm5"
Jul 07 00:08:28.494117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:28.494079    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:28.594211 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:28.594167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:30.411761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:30.411722    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:44.988342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.988304    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"]
Jul 07 00:08:44.988889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.988692    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" containerName="authorino"
Jul 07 00:08:44.988889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.988703    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" containerName="authorino"
Jul 07 00:08:44.988889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.988757    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac0f36e-60ce-4765-bc47-4a52ea3c8ebf" containerName="authorino"
Jul 07 00:08:44.992048 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.992025    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:44.994761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.994733    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kvp5g\""
Jul 07 00:08:44.995871 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.995852    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 07 00:08:44.995985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.995874    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 07 00:08:44.995985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:44.995932    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 07 00:08:45.001168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.001145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"]
Jul 07 00:08:45.022130 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.022096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:45.151527 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a59c2-3fa1-403b-8717-a364d7824792-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.151703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.151703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.151703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.151863 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151740    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mkjq\" (UniqueName: \"kubernetes.io/projected/da3a59c2-3fa1-403b-8717-a364d7824792-kube-api-access-6mkjq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.151863 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.151842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252266    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mkjq\" (UniqueName: \"kubernetes.io/projected/da3a59c2-3fa1-403b-8717-a364d7824792-kube-api-access-6mkjq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252605 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252605 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a59c2-3fa1-403b-8717-a364d7824792-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252605 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252419    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252605 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252460    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252839 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252690    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252908 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252860    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.252908 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.252883    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.255221 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.255199    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da3a59c2-3fa1-403b-8717-a364d7824792-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.255779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.255753    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da3a59c2-3fa1-403b-8717-a364d7824792-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.261382 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.261357    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mkjq\" (UniqueName: \"kubernetes.io/projected/da3a59c2-3fa1-403b-8717-a364d7824792-kube-api-access-6mkjq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hdnq7\" (UID: \"da3a59c2-3fa1-403b-8717-a364d7824792\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.303439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.303403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:45.442229 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:45.442198    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"]
Jul 07 00:08:45.443941 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:08:45.443911    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3a59c2_3fa1_403b_8717_a364d7824792.slice/crio-68eab106f7657d949beb2b65a483bfa05429000f3aec00589239c56b665f92e0 WatchSource:0}: Error finding container 68eab106f7657d949beb2b65a483bfa05429000f3aec00589239c56b665f92e0: Status 404 returned error can't find the container with id 68eab106f7657d949beb2b65a483bfa05429000f3aec00589239c56b665f92e0
Jul 07 00:08:46.191360 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:46.191275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7" event={"ID":"da3a59c2-3fa1-403b-8717-a364d7824792","Type":"ContainerStarted","Data":"68eab106f7657d949beb2b65a483bfa05429000f3aec00589239c56b665f92e0"}
Jul 07 00:08:47.197463 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:47.197421    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7" event={"ID":"da3a59c2-3fa1-403b-8717-a364d7824792","Type":"ContainerStarted","Data":"8548f89cfacbb1a9093516540da21aa9ab64b920f95234fafbb69f4894913efd"}
Jul 07 00:08:47.225652 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:47.225597    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7" podStartSLOduration=2.059072428 podStartE2EDuration="3.225578714s" totalImagesPullingTime="1.166506286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:08:47.223568314 +0000 UTC m=+712.548571598" watchObservedRunningTime="2026-07-07 00:08:47.225578714 +0000 UTC m=+712.550581997"
Jul 07 00:08:53.419742 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:53.419704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:08:55.304537 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:55.304505    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:55.305007 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:55.304565    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:55.317106 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:55.317082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:08:56.252285 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:08:56.252253    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hdnq7"
Jul 07 00:09:03.578558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.578525    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"]
Jul 07 00:09:03.582558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.582535    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.585322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.585281    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 07 00:09:03.594091 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.594065    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"]
Jul 07 00:09:03.624480 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624448    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.624639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624496    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.624639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.624639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624551    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.624639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624581    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kf68\" (UniqueName: \"kubernetes.io/projected/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-kube-api-access-4kf68\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.624639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.624631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.630622 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.630590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:09:03.725624 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.725814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725667    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.725814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.725814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725735    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.725814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.725814 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.725804    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kf68\" (UniqueName: \"kubernetes.io/projected/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-kube-api-access-4kf68\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.726090 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.726063    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.726149 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.726121    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.726353 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.726332    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.728055 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.728026    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.728372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.728351    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.733484 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.733458    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kf68\" (UniqueName: \"kubernetes.io/projected/c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5-kube-api-access-4kf68\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp\" (UID: \"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:03.894190 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:03.894153    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:04.033550 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:04.033509    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"]
Jul 07 00:09:04.035172 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:09:04.035139    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6bb2b2f_9ae5_4ff5_b3c9_1bfc78c33cd5.slice/crio-192a6d483c32729b2414def55a1df653df5e9301c289f34b67e04f7b93574603 WatchSource:0}: Error finding container 192a6d483c32729b2414def55a1df653df5e9301c289f34b67e04f7b93574603: Status 404 returned error can't find the container with id 192a6d483c32729b2414def55a1df653df5e9301c289f34b67e04f7b93574603
Jul 07 00:09:04.281824 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:04.281792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp" event={"ID":"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5","Type":"ContainerStarted","Data":"5b9feb493bcb9a946640acac64afcfaf914dfb19786aabff76412c5fc506a9f3"}
Jul 07 00:09:04.281976 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:04.281850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp" event={"ID":"c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5","Type":"ContainerStarted","Data":"192a6d483c32729b2414def55a1df653df5e9301c289f34b67e04f7b93574603"}
Jul 07 00:09:04.300577 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:04.300529    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp" podStartSLOduration=1.114457734 podStartE2EDuration="1.300512006s" totalImagesPullingTime="186.054272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:09:04.297751202 +0000 UTC m=+729.622754486" watchObservedRunningTime="2026-07-07 00:09:04.300512006 +0000 UTC m=+729.625515289"
Jul 07 00:09:09.521321 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:09.521267    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:09:13.895145 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:13.895095    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:13.895697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:13.895175    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:13.908038 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:13.908011    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:14.339620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:14.339534    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp"
Jul 07 00:09:29.218202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:09:29.218161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:26.545936 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:26.545843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:26.646015 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:26.645970    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:27.451620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.451581    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:10:27.451829 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.451801    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" podUID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" containerName="authorino" containerID="cri-o://fefb443041f4b4363a27b66f4058e48406ff4f38b52606e538ea95875c5bb33d" gracePeriod=30
Jul 07 00:10:27.694761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.694727    2583 generic.go:356] "Generic (PLEG): container finished" podID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" containerID="fefb443041f4b4363a27b66f4058e48406ff4f38b52606e538ea95875c5bb33d" exitCode=0
Jul 07 00:10:27.695089 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.694797    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" event={"ID":"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b","Type":"ContainerDied","Data":"fefb443041f4b4363a27b66f4058e48406ff4f38b52606e538ea95875c5bb33d"}
Jul 07 00:10:27.695089 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.694843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-84bb5bc755-5w7vk" event={"ID":"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b","Type":"ContainerDied","Data":"2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c"}
Jul 07 00:10:27.695089 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.694855    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cd8393e22838b903fb6ac621739b83ab1c2227a4adda2fe0a7e0300fac2502c"
Jul 07 00:10:27.705738 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.705674    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:10:27.817827 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.817792    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert\") pod \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") "
Jul 07 00:10:27.818025 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.817891    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zn24\" (UniqueName: \"kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24\") pod \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\" (UID: \"e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b\") "
Jul 07 00:10:27.819880 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.819851    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24" (OuterVolumeSpecName: "kube-api-access-4zn24") pod "e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" (UID: "e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b"). InnerVolumeSpecName "kube-api-access-4zn24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:10:27.827805 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.827779    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" (UID: "e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:10:27.872808 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.872772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:27.918996 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.918964    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zn24\" (UniqueName: \"kubernetes.io/projected/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-kube-api-access-4zn24\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:10:27.918996 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.918994    2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b-tls-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:10:27.973351 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:27.973225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:28.699519 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:28.699488    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-84bb5bc755-5w7vk"
Jul 07 00:10:28.721382 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:28.721354    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:10:28.725218 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:28.725196    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-84bb5bc755-5w7vk"]
Jul 07 00:10:29.158660 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:29.158629    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" path="/var/lib/kubelet/pods/e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b/volumes"
Jul 07 00:10:34.802945 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:34.802911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:10:34.901609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:10:34.901575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:11:39.970344 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:11:39.970274    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:11:41.498483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:11:41.498446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:11:55.189453 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:11:55.189424    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:11:55.196724 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:11:55.196697    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:12:21.390019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:21.389972    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:21.486916 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:21.486874    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:21.623766 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:21.623727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:27.727257 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:27.727225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:29.184960 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:29.184913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:33.988482 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:33.988444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:12:34.054029 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:12:34.053998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:13:30.516319 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:13:30.516217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:13:30.574456 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:13:30.574423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:13:55.198797 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:13:55.198766    2583 scope.go:117] "RemoveContainer" containerID="fefb443041f4b4363a27b66f4058e48406ff4f38b52606e538ea95875c5bb33d"
Jul 07 00:15:00.138448 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.138413    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:15:00.138932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.138768    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" containerName="authorino"
Jul 07 00:15:00.138932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.138781    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" containerName="authorino"
Jul 07 00:15:00.138932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.138852    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e17fc9bd-577d-45a3-9b4a-1b5b4ce3587b" containerName="authorino"
Jul 07 00:15:00.141761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.141735    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:15:00.145178 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.145146    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lq9jt\""
Jul 07 00:15:00.145414 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.145398    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 00:15:00.146218 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.146201    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 00:15:00.167372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.167331    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:15:00.313102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.313058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqzqp\" (UniqueName: \"kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp\") pod \"maas-api-key-cleanup-29723055-jpzkf\" (UID: \"9c5f69f9-bd29-4e4c-9974-ada2128b3f08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:15:00.414240 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.414155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqzqp\" (UniqueName: \"kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp\") pod \"maas-api-key-cleanup-29723055-jpzkf\" (UID: \"9c5f69f9-bd29-4e4c-9974-ada2128b3f08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:15:00.422818 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.422784    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqzqp\" (UniqueName: \"kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp\") pod \"maas-api-key-cleanup-29723055-jpzkf\" (UID: \"9c5f69f9-bd29-4e4c-9974-ada2128b3f08\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:15:00.452627 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.452596    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:15:00.589851 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.589829    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:15:00.591807 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:15:00.591783    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c5f69f9_bd29_4e4c_9974_ada2128b3f08.slice/crio-7fd46d5912396ad19632dbd4800b9ecca9926a477abcfdb65d3625c5736bf0e6 WatchSource:0}: Error finding container 7fd46d5912396ad19632dbd4800b9ecca9926a477abcfdb65d3625c5736bf0e6: Status 404 returned error can't find the container with id 7fd46d5912396ad19632dbd4800b9ecca9926a477abcfdb65d3625c5736bf0e6
Jul 07 00:15:00.593765 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:00.593748    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:15:01.022154 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:01.022121    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" event={"ID":"9c5f69f9-bd29-4e4c-9974-ada2128b3f08","Type":"ContainerStarted","Data":"7fd46d5912396ad19632dbd4800b9ecca9926a477abcfdb65d3625c5736bf0e6"}
Jul 07 00:15:04.038591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:04.038556    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" event={"ID":"9c5f69f9-bd29-4e4c-9974-ada2128b3f08","Type":"ContainerStarted","Data":"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"}
Jul 07 00:15:04.059679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:04.059626    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" podStartSLOduration=1.9367450750000001 podStartE2EDuration="4.059608031s" totalImagesPullingTime="2.122862956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:04.058149546 +0000 UTC m=+1089.383152831" watchObservedRunningTime="2026-07-07 00:15:04.059608031 +0000 UTC m=+1089.384611317"
Jul 07 00:15:37.663688 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:15:37.663652    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:16:00.010389 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:00.010349    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:16:00.010900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:00.010624    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" podUID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" containerName="cleanup" containerID="cri-o://104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044" gracePeriod=30
Jul 07 00:16:28.642404 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:28.642364    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:16:28.642909 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:28.642583    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" podUID="0358cb66-2534-483e-a56e-e099847da9ce" containerName="manager" containerID="cri-o://6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688" gracePeriod=10
Jul 07 00:16:30.192004 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.191978    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:16:30.247975 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.247938    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqzqp\" (UniqueName: \"kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp\") pod \"9c5f69f9-bd29-4e4c-9974-ada2128b3f08\" (UID: \"9c5f69f9-bd29-4e4c-9974-ada2128b3f08\") "
Jul 07 00:16:30.250111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.250078    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp" (OuterVolumeSpecName: "kube-api-access-wqzqp") pod "9c5f69f9-bd29-4e4c-9974-ada2128b3f08" (UID: "9c5f69f9-bd29-4e4c-9974-ada2128b3f08"). InnerVolumeSpecName "kube-api-access-wqzqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:16:30.349161 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.349068    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqzqp\" (UniqueName: \"kubernetes.io/projected/9c5f69f9-bd29-4e4c-9974-ada2128b3f08-kube-api-access-wqzqp\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:16:30.458455 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.458418    2583 generic.go:356] "Generic (PLEG): container finished" podID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" containerID="104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044" exitCode=137
Jul 07 00:16:30.458632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.458465    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"
Jul 07 00:16:30.458632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.458498    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" event={"ID":"9c5f69f9-bd29-4e4c-9974-ada2128b3f08","Type":"ContainerDied","Data":"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"}
Jul 07 00:16:30.458632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.458553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf" event={"ID":"9c5f69f9-bd29-4e4c-9974-ada2128b3f08","Type":"ContainerDied","Data":"7fd46d5912396ad19632dbd4800b9ecca9926a477abcfdb65d3625c5736bf0e6"}
Jul 07 00:16:30.458632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.458576    2583 scope.go:117] "RemoveContainer" containerID="104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"
Jul 07 00:16:30.467495 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.467475    2583 scope.go:117] "RemoveContainer" containerID="104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"
Jul 07 00:16:30.467726 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:16:30.467709    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044\": container with ID starting with 104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044 not found: ID does not exist" containerID="104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"
Jul 07 00:16:30.467776 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.467734    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044"} err="failed to get container status \"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044\": rpc error: code = NotFound desc = could not find container \"104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044\": container with ID starting with 104a778a5014d2b69ce71928057a7c1f8a747414cbaa61040608eb3403f4e044 not found: ID does not exist"
Jul 07 00:16:30.480569 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.480543    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:16:30.484399 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:30.484378    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29723055-jpzkf"]
Jul 07 00:16:31.159287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.159250    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" path="/var/lib/kubelet/pods/9c5f69f9-bd29-4e4c-9974-ada2128b3f08/volumes"
Jul 07 00:16:31.211455 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.211429    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:16:31.257116 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.257085    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9mph\" (UniqueName: \"kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph\") pod \"0358cb66-2534-483e-a56e-e099847da9ce\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") "
Jul 07 00:16:31.257283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.257124    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume\") pod \"0358cb66-2534-483e-a56e-e099847da9ce\" (UID: \"0358cb66-2534-483e-a56e-e099847da9ce\") "
Jul 07 00:16:31.257711 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.257625    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0358cb66-2534-483e-a56e-e099847da9ce" (UID: "0358cb66-2534-483e-a56e-e099847da9ce"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:16:31.259391 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.259358    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph" (OuterVolumeSpecName: "kube-api-access-w9mph") pod "0358cb66-2534-483e-a56e-e099847da9ce" (UID: "0358cb66-2534-483e-a56e-e099847da9ce"). InnerVolumeSpecName "kube-api-access-w9mph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:16:31.358778 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.358686    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9mph\" (UniqueName: \"kubernetes.io/projected/0358cb66-2534-483e-a56e-e099847da9ce-kube-api-access-w9mph\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:16:31.358778 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.358714    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0358cb66-2534-483e-a56e-e099847da9ce-extensions-socket-volume\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:16:31.465042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.465009    2583 generic.go:356] "Generic (PLEG): container finished" podID="0358cb66-2534-483e-a56e-e099847da9ce" containerID="6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688" exitCode=0
Jul 07 00:16:31.465249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.465061    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"
Jul 07 00:16:31.465249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.465099    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" event={"ID":"0358cb66-2534-483e-a56e-e099847da9ce","Type":"ContainerDied","Data":"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"}
Jul 07 00:16:31.465249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.465152    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx" event={"ID":"0358cb66-2534-483e-a56e-e099847da9ce","Type":"ContainerDied","Data":"7cc40475f95e7b48a34f81067849415774ca806f9c49f5f278878807e3adf763"}
Jul 07 00:16:31.465249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.465174    2583 scope.go:117] "RemoveContainer" containerID="6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"
Jul 07 00:16:31.475099 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.475082    2583 scope.go:117] "RemoveContainer" containerID="6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"
Jul 07 00:16:31.475399 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:16:31.475378    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688\": container with ID starting with 6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688 not found: ID does not exist" containerID="6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"
Jul 07 00:16:31.475475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.475408    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688"} err="failed to get container status \"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688\": rpc error: code = NotFound desc = could not find container \"6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688\": container with ID starting with 6444afd01ede56d55c8f2af988b29c0c7250fc8398bd0885a7fdcc9f48cba688 not found: ID does not exist"
Jul 07 00:16:31.487614 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.487584    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:16:31.493556 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:31.493531    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4znx"]
Jul 07 00:16:33.158132 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:33.158086    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0358cb66-2534-483e-a56e-e099847da9ce" path="/var/lib/kubelet/pods/0358cb66-2534-483e-a56e-e099847da9ce/volumes"
Jul 07 00:16:55.218548 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:55.218516    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:16:55.224111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:55.224084    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:16:57.068163 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:57.068123    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:16:57.122630 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:16:57.122592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:17:17.444813 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:17.444772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:17:29.843184 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843151    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"]
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843559    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0358cb66-2534-483e-a56e-e099847da9ce" containerName="manager"
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843573    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0358cb66-2534-483e-a56e-e099847da9ce" containerName="manager"
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843596    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" containerName="cleanup"
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843602    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" containerName="cleanup"
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843662    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0358cb66-2534-483e-a56e-e099847da9ce" containerName="manager"
Jul 07 00:17:29.843716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.843669    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c5f69f9-bd29-4e4c-9974-ada2128b3f08" containerName="cleanup"
Jul 07 00:17:29.846667 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.846648    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.850571 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.850551    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-q4b5q\""
Jul 07 00:17:29.869093 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.869058    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.869254 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.869132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnc25\" (UniqueName: \"kubernetes.io/projected/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-kube-api-access-vnc25\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.870189 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.870168    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"]
Jul 07 00:17:29.970671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.970627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.970865 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.970683    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnc25\" (UniqueName: \"kubernetes.io/projected/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-kube-api-access-vnc25\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.971046 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.971023    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:29.980019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:29.979983    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnc25\" (UniqueName: \"kubernetes.io/projected/0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b-kube-api-access-vnc25\") pod \"kuadrant-operator-controller-manager-86878c769f-29nx5\" (UID: \"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:30.156988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.156952    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:30.301922 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.301894    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"]
Jul 07 00:17:30.304068 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:17:30.304040    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0adbe9f5_f9f6_48fd_9f21_42a1687c8e7b.slice/crio-cb3f07cb2f2b822b01b62c65f2100f2479b43ef4c4bcb6118cdb6e960c3aa626 WatchSource:0}: Error finding container cb3f07cb2f2b822b01b62c65f2100f2479b43ef4c4bcb6118cdb6e960c3aa626: Status 404 returned error can't find the container with id cb3f07cb2f2b822b01b62c65f2100f2479b43ef4c4bcb6118cdb6e960c3aa626
Jul 07 00:17:30.747452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.747418    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5" event={"ID":"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b","Type":"ContainerStarted","Data":"b29b842c2b0753bd071c916e0542f6e163061e1ca5ef0ed15431375accad88e8"}
Jul 07 00:17:30.747619 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.747476    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5" event={"ID":"0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b","Type":"ContainerStarted","Data":"cb3f07cb2f2b822b01b62c65f2100f2479b43ef4c4bcb6118cdb6e960c3aa626"}
Jul 07 00:17:30.747619 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.747513    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:17:30.770906 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:30.770841    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5" podStartSLOduration=1.770819447 podStartE2EDuration="1.770819447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:30.766541407 +0000 UTC m=+1236.091544692" watchObservedRunningTime="2026-07-07 00:17:30.770819447 +0000 UTC m=+1236.095822732"
Jul 07 00:17:41.754897 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:17:41.754861    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-29nx5"
Jul 07 00:18:25.065223 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:18:25.065187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:18:26.893563 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:18:26.893531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:21:55.247836 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:21:55.247807    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:21:55.255123 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:21:55.255101    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:22:57.035690 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:22:57.035656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:16.938693 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:16.938658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:24.446124 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:24.446086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:32.704779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:32.704743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:37.870591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:37.870551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:40.887324 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:40.887264    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:40.988181 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:40.988143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:41.360617 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:41.360533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:47.363607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.363497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:47.433719 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.433678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:47.531762 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.531722    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:23:47.536120 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.536091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.539191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.539166    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-58bac345-openshift-default-dockercfg-p8v2d\""
Jul 07 00:23:47.557559 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.557530    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:23:47.561274 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.561247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:47.637838 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.637791    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.637860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.637889    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.637930    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.637956    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlb7f\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.638072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.638110    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.638129    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.638332 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.638207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.660561 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.660523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:47.739648 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739648 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739783    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlb7f\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.739896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.739889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740211 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740211 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740079    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740211 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740167    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740399 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740214    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740399 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740240    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740573 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740550    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.740634 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.740554    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.742357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.742338    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.742503 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.742487    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.754637 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.754614    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlb7f\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.756015 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.755994    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token\") pod \"e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") " pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:47.849748 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:47.849710    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:48.007757 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.007727    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:23:48.009388 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:23:48.009358    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51866e9_a8c9_4daf_9ea8_b7272a906ddf.slice/crio-ad4a67481d5f69449dc06304ab0f0f85cd579852f9ee3660001f5c1177fc91d7 WatchSource:0}: Error finding container ad4a67481d5f69449dc06304ab0f0f85cd579852f9ee3660001f5c1177fc91d7: Status 404 returned error can't find the container with id ad4a67481d5f69449dc06304ab0f0f85cd579852f9ee3660001f5c1177fc91d7
Jul 07 00:23:48.011520 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.011481    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:23:48.011587 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.011556    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:23:48.011627 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.011595    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:23:48.594286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.594233    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" event={"ID":"c51866e9-a8c9-4daf-9ea8-b7272a906ddf","Type":"ContainerStarted","Data":"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"}
Jul 07 00:23:48.594785 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.594319    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" event={"ID":"c51866e9-a8c9-4daf-9ea8-b7272a906ddf","Type":"ContainerStarted","Data":"ad4a67481d5f69449dc06304ab0f0f85cd579852f9ee3660001f5c1177fc91d7"}
Jul 07 00:23:48.615929 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.615881    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" podStartSLOduration=1.615865586 podStartE2EDuration="1.615865586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:23:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:23:48.614611217 +0000 UTC m=+1613.939614516" watchObservedRunningTime="2026-07-07 00:23:48.615865586 +0000 UTC m=+1613.940868869"
Jul 07 00:23:48.711963 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.711922    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:48.850812 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.850719    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:48.922157 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:48.922120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:49.020237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:49.020201    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:49.121407 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:49.121375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:49.422279 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:49.422198    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:49.522137 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:49.522105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:49.854975 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:49.854900    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:50.604851 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:50.604819    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:50.605974 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:50.605946    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:23:51.189167 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:51.189132    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:53.858875 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:53.858814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:54.155451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:54.155417    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:23:54.155970 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:54.155939    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" podUID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" containerName="istio-proxy" containerID="cri-o://f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5" gracePeriod=30
Jul 07 00:23:54.186519 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:54.186485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:54.279756 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:54.279721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:54.382887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:54.382855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:57.397150 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:57.397117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:23:59.951827 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:23:59.951790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.113506 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.113457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.137758 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.137732    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:00.140806 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.140775    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.143639 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.143616    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6738905c-openshift-default-dockercfg-drnpk\""
Jul 07 00:24:00.152836 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.152806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.152948 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.152860    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.152948 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.152893    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.152948 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.152935    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153086 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153014    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4sr7\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153086 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153045    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153086 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153072    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153095    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153128    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.153326 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.153309    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:00.218102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.218023    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:00.253999 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.253942    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254203 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254203 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254104    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254203 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254185    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254402 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254233    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4sr7\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254402 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254263    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254402 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254402 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254360    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254402 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254383    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254927 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254897    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.254927 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.254921    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.255252 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.255205    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.255405 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.255253    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.255405 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.255273    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.257493 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.257418    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.257776 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.257752    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.262867 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.262839    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.262985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.262968    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4sr7\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7\") pod \"e2e-ait-6738905c-openshift-default-54d6f58669-vtc94\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") " pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.310817 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.310792    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:24:00.355129 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355094    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355145    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355193    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355235    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355263    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355550 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355342    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355550 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355365    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355550 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355510    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:00.355718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355544    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:00.355718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355570    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlb7f\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355629    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy\") pod \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\" (UID: \"c51866e9-a8c9-4daf-9ea8-b7272a906ddf\") "
Jul 07 00:24:00.355718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355636    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:00.355935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355722    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:24:00.355935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355893    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.355935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355910    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.355935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355924    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.356147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.355937    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.356265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.356236    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data" (OuterVolumeSpecName: "istio-data") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:00.357458 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.357425    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token" (OuterVolumeSpecName: "istio-token") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:00.357559 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.357501    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:24:00.357634 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.357563    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f" (OuterVolumeSpecName: "kube-api-access-tlb7f") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "kube-api-access-tlb7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:00.357634 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.357617    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c51866e9-a8c9-4daf-9ea8-b7272a906ddf" (UID: "c51866e9-a8c9-4daf-9ea8-b7272a906ddf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:00.456683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.456645    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.456683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.456676    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.456683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.456684    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlb7f\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-kube-api-access-tlb7f\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.456683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.456693    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.456966 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.456703    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c51866e9-a8c9-4daf-9ea8-b7272a906ddf-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:00.481611 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.481534    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:00.629577 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.629549    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:00.632267 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:24:00.632229    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4705d4_f827_41d2_8742_5673c430127b.slice/crio-c6e04de9dabd449b41b4a1ab1656815c959d84ccc5095d311bf6e4ed1126538a WatchSource:0}: Error finding container c6e04de9dabd449b41b4a1ab1656815c959d84ccc5095d311bf6e4ed1126538a: Status 404 returned error can't find the container with id c6e04de9dabd449b41b4a1ab1656815c959d84ccc5095d311bf6e4ed1126538a
Jul 07 00:24:00.634579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.634546    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:00.634664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.634614    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:00.634664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.634643    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:00.662967 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.662939    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" event={"ID":"ea4705d4-f827-41d2-8742-5673c430127b","Type":"ContainerStarted","Data":"c6e04de9dabd449b41b4a1ab1656815c959d84ccc5095d311bf6e4ed1126538a"}
Jul 07 00:24:00.664050 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.664028    2583 generic.go:356] "Generic (PLEG): container finished" podID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" containerID="f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5" exitCode=0
Jul 07 00:24:00.664158 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.664072    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"
Jul 07 00:24:00.664158 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.664099    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" event={"ID":"c51866e9-a8c9-4daf-9ea8-b7272a906ddf","Type":"ContainerDied","Data":"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"}
Jul 07 00:24:00.664158 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.664137    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv" event={"ID":"c51866e9-a8c9-4daf-9ea8-b7272a906ddf","Type":"ContainerDied","Data":"ad4a67481d5f69449dc06304ab0f0f85cd579852f9ee3660001f5c1177fc91d7"}
Jul 07 00:24:00.664259 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.664160    2583 scope.go:117] "RemoveContainer" containerID="f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"
Jul 07 00:24:00.681282 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.681260    2583 scope.go:117] "RemoveContainer" containerID="f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"
Jul 07 00:24:00.681620 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:24:00.681599    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5\": container with ID starting with f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5 not found: ID does not exist" containerID="f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"
Jul 07 00:24:00.681687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.681633    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5"} err="failed to get container status \"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5\": rpc error: code = NotFound desc = could not find container \"f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5\": container with ID starting with f58661d35c5e034f25422b6707717abe2b9d19df2aa1755a7624d35b7e20dbd5 not found: ID does not exist"
Jul 07 00:24:00.690952 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.690923    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:24:00.695530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:00.695503    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-58bac345-openshift-default-5457d74f97-d5wkv"]
Jul 07 00:24:01.158798 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.158764    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" path="/var/lib/kubelet/pods/c51866e9-a8c9-4daf-9ea8-b7272a906ddf/volumes"
Jul 07 00:24:01.406964 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.406862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:01.491659 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.491627    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:01.573590 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.573551    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:01.670051 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.669944    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" event={"ID":"ea4705d4-f827-41d2-8742-5673c430127b","Type":"ContainerStarted","Data":"1fe6c59e92da7044b752e6f53f90d46bb8cf7704fe5472d726953c7e4cbec987"}
Jul 07 00:24:01.691945 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:01.691892    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" podStartSLOduration=1.691875833 podStartE2EDuration="1.691875833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:01.690566474 +0000 UTC m=+1627.015569758" watchObservedRunningTime="2026-07-07 00:24:01.691875833 +0000 UTC m=+1627.016879116"
Jul 07 00:24:02.045283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.045186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:02.142202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.142167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:02.481955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.481909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:02.486579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.486556    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:02.675473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.675438    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:02.676364 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:02.676347    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:03.237679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:03.237646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:06.635322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:06.635002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:06.975556 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:06.975461    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:12.685071 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.684988    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:12.685578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.685199    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" podUID="ea4705d4-f827-41d2-8742-5673c430127b" containerName="istio-proxy" containerID="cri-o://1fe6c59e92da7044b752e6f53f90d46bb8cf7704fe5472d726953c7e4cbec987" gracePeriod=30
Jul 07 00:24:12.707078 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.707044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:12.724883 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.724798    2583 generic.go:356] "Generic (PLEG): container finished" podID="ea4705d4-f827-41d2-8742-5673c430127b" containerID="1fe6c59e92da7044b752e6f53f90d46bb8cf7704fe5472d726953c7e4cbec987" exitCode=0
Jul 07 00:24:12.725024 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.724872    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" event={"ID":"ea4705d4-f827-41d2-8742-5673c430127b","Type":"ContainerDied","Data":"1fe6c59e92da7044b752e6f53f90d46bb8cf7704fe5472d726953c7e4cbec987"}
Jul 07 00:24:12.818677 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.818647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:12.830560 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.830536    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:12.884027 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.883986    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4sr7\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884041    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884097    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884127    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884212    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884470 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884250    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884470 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884278    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884470 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884341    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884470 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo\") pod \"ea4705d4-f827-41d2-8742-5673c430127b\" (UID: \"ea4705d4-f827-41d2-8742-5673c430127b\") "
Jul 07 00:24:12.884663 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884531    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:24:12.884905 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884743    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea4705d4-f827-41d2-8742-5673c430127b-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.885001 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.884955    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:12.885309 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.885141    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:12.885399 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.885359    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:12.885523 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.885502    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:12.887768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.887349    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:24:12.888540 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.888506    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:12.888644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.888621    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:12.888931 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.888898    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7" (OuterVolumeSpecName: "kube-api-access-j4sr7") pod "ea4705d4-f827-41d2-8742-5673c430127b" (UID: "ea4705d4-f827-41d2-8742-5673c430127b"). InnerVolumeSpecName "kube-api-access-j4sr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986050    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986097    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986108    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986116    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986125    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea4705d4-f827-41d2-8742-5673c430127b-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986133    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4sr7\" (UniqueName: \"kubernetes.io/projected/ea4705d4-f827-41d2-8742-5673c430127b-kube-api-access-j4sr7\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986141    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:12.986156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:12.986149    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea4705d4-f827-41d2-8742-5673c430127b-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:13.731201 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:13.731167    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"
Jul 07 00:24:13.731201 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:13.731183    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94" event={"ID":"ea4705d4-f827-41d2-8742-5673c430127b","Type":"ContainerDied","Data":"c6e04de9dabd449b41b4a1ab1656815c959d84ccc5095d311bf6e4ed1126538a"}
Jul 07 00:24:13.731728 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:13.731250    2583 scope.go:117] "RemoveContainer" containerID="1fe6c59e92da7044b752e6f53f90d46bb8cf7704fe5472d726953c7e4cbec987"
Jul 07 00:24:13.750439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:13.750410    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:13.755598 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:13.755573    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6738905c-openshift-default-54d6f58669-vtc94"]
Jul 07 00:24:15.159014 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:15.158978    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4705d4-f827-41d2-8742-5673c430127b" path="/var/lib/kubelet/pods/ea4705d4-f827-41d2-8742-5673c430127b/volumes"
Jul 07 00:24:15.263127 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:15.263094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:18.481043 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.481008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:18.580703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.580650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:18.655439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.655402    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:18.656109 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656080    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4705d4-f827-41d2-8742-5673c430127b" containerName="istio-proxy"
Jul 07 00:24:18.656109 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656110    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4705d4-f827-41d2-8742-5673c430127b" containerName="istio-proxy"
Jul 07 00:24:18.656348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656137    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" containerName="istio-proxy"
Jul 07 00:24:18.656348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656145    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" containerName="istio-proxy"
Jul 07 00:24:18.656348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656231    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c51866e9-a8c9-4daf-9ea8-b7272a906ddf" containerName="istio-proxy"
Jul 07 00:24:18.656348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.656242    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4705d4-f827-41d2-8742-5673c430127b" containerName="istio-proxy"
Jul 07 00:24:18.659357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.659334    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.662392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.662371    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6c06fb72-openshift-default-dockercfg-rrwms\""
Jul 07 00:24:18.670519 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.670492    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:18.696914 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.696869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:18.738219 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738117    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738219 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738175    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738256    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738321    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738362    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738495    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47xpr\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.738644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.738600    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.797518 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.797481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:18.839580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839546    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839585    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839644    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839749    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839777    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47xpr\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.839837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.839802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.840205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.840017    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.840205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.840069    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.840205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.840180    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.840205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.840195    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.840558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.840537    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.841930 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.841913    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.842221 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.842202    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.848313 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.848269    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.848497 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.848480    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47xpr\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr\") pod \"e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") " pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:18.971242 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:18.971206    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:19.121920 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.121886    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:19.123305 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:24:19.123266    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e2981b_5f44_4390_a5a5_1b3fa53a03ea.slice/crio-231978917eb49625a3e7f2476659e78109fa0ff3cf3b86eee3bb2a15b718baa6 WatchSource:0}: Error finding container 231978917eb49625a3e7f2476659e78109fa0ff3cf3b86eee3bb2a15b718baa6: Status 404 returned error can't find the container with id 231978917eb49625a3e7f2476659e78109fa0ff3cf3b86eee3bb2a15b718baa6
Jul 07 00:24:19.125621 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.125572    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:19.125706 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.125676    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:19.125745 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.125727    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:24:19.763506 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.763465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" event={"ID":"51e2981b-5f44-4390-a5a5-1b3fa53a03ea","Type":"ContainerStarted","Data":"dd20e54c45aed27ceef0892d8938115f45d47853aeefec3905db2bd91778e301"}
Jul 07 00:24:19.763918 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.763527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" event={"ID":"51e2981b-5f44-4390-a5a5-1b3fa53a03ea","Type":"ContainerStarted","Data":"231978917eb49625a3e7f2476659e78109fa0ff3cf3b86eee3bb2a15b718baa6"}
Jul 07 00:24:19.782985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.782924    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" podStartSLOduration=1.782902409 podStartE2EDuration="1.782902409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:24:19.782578797 +0000 UTC m=+1645.107582082" watchObservedRunningTime="2026-07-07 00:24:19.782902409 +0000 UTC m=+1645.107905694"
Jul 07 00:24:19.877863 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.877797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:19.962355 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.962284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:19.972210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:19.972174    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:20.043055 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:20.042961    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.135184 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:20.135148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.561793 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:20.561735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.660261 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:20.660221    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:20.977140 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:20.977110    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:21.772702 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:21.772669    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:21.773609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:21.773585    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:22.108309 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:22.108265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:24.357431 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.357382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:24.518983 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.518943    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:24.544430 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.544388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:24.645468 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.645434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:24.763215 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.763183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:24.786673 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:24.786633    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" podUID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" containerName="istio-proxy" containerID="cri-o://dd20e54c45aed27ceef0892d8938115f45d47853aeefec3905db2bd91778e301" gracePeriod=30
Jul 07 00:24:26.164047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:26.164009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:30.314153 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.314120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:30.413721 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.413683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:30.802489 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.802394    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:30.818448 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.818408    2583 generic.go:356] "Generic (PLEG): container finished" podID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" containerID="dd20e54c45aed27ceef0892d8938115f45d47853aeefec3905db2bd91778e301" exitCode=0
Jul 07 00:24:30.818633 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.818479    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" event={"ID":"51e2981b-5f44-4390-a5a5-1b3fa53a03ea","Type":"ContainerDied","Data":"dd20e54c45aed27ceef0892d8938115f45d47853aeefec3905db2bd91778e301"}
Jul 07 00:24:30.853359 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.853335    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:30.899509 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.899416    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:30.952273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952237    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952274    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952317    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952373    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952407    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952467    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952490    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47xpr\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952514    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952853 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952605    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket\") pod \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\" (UID: \"51e2981b-5f44-4390-a5a5-1b3fa53a03ea\") "
Jul 07 00:24:30.952919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952576    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:30.952919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952728    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:30.953004 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952747    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:24:30.953004 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.952930    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:30.953285 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.953254    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:30.954764 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.954739    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:24:30.954860 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.954745    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:30.954917 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.954896    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:24:30.954954 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.954920    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr" (OuterVolumeSpecName: "kube-api-access-47xpr") pod "51e2981b-5f44-4390-a5a5-1b3fa53a03ea" (UID: "51e2981b-5f44-4390-a5a5-1b3fa53a03ea"). InnerVolumeSpecName "kube-api-access-47xpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:24:30.966079 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:30.966047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:31.053384 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053344    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053384 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053377    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053384 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053387    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053396    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47xpr\" (UniqueName: \"kubernetes.io/projected/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-kube-api-access-47xpr\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053407    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053415    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053423    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053431    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.053626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.053439    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51e2981b-5f44-4390-a5a5-1b3fa53a03ea-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:24:31.824050 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.824025    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"
Jul 07 00:24:31.824579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.824063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv" event={"ID":"51e2981b-5f44-4390-a5a5-1b3fa53a03ea","Type":"ContainerDied","Data":"231978917eb49625a3e7f2476659e78109fa0ff3cf3b86eee3bb2a15b718baa6"}
Jul 07 00:24:31.824579 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.824118    2583 scope.go:117] "RemoveContainer" containerID="dd20e54c45aed27ceef0892d8938115f45d47853aeefec3905db2bd91778e301"
Jul 07 00:24:31.862838 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.862802    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:31.867915 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:31.867890    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6c06fb72-openshift-default-7f7d649855-8cppv"]
Jul 07 00:24:32.523236 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:32.523197    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:33.158539 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:33.158486    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" path="/var/lib/kubelet/pods/51e2981b-5f44-4390-a5a5-1b3fa53a03ea/volumes"
Jul 07 00:24:38.810511 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:38.810472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:38.931844 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:38.931803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:39.042344 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:39.042305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:39.140043 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:39.140009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:39.283344 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:39.283311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:24:39.578187 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:24:39.578067    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:05.610613 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:05.610577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:11.727931 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:11.727893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:11.802235 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:11.802200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:11.882876 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:11.882845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:11.984100 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:11.983997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:27.904011 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:27.903908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:33.860445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:33.860407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:33.964626 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:33.964581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:34.077037 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:34.077004    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:44.593240 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:44.593192    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:45.425653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:45.425616    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:51.245228 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:51.245183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:51.354672 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:51.354632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:51.463692 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:51.463632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:51.631661 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:51.631631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:52.336686 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:52.336643    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:52.431250 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:52.431196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:52.530509 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:52.530460    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:25:54.179394 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:25:54.179358    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:00.097255 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:00.097215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:00.179864 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:00.179833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:00.798807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:00.798772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:07.095217 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.095181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:07.192406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.192374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:07.327525 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.327483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:07.645971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.645932    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:07.646385 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.646371    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" containerName="istio-proxy"
Jul 07 00:26:07.646442 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.646387    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" containerName="istio-proxy"
Jul 07 00:26:07.646480 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.646448    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e2981b-5f44-4390-a5a5-1b3fa53a03ea" containerName="istio-proxy"
Jul 07 00:26:07.649461 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.649444    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.652408 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.652389    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9eadf20a-openshift-default-dockercfg-4wphq\""
Jul 07 00:26:07.666474 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.666443    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:07.678510 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.678474    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:07.729958 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.729917    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.729969    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730036    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb9qh\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730158    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730243 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730275    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.730732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.730328    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831418 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831373    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831599    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831842 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831842 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831842 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831748    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb9qh\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.831842 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831792    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.832042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.831948    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.832042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.832010    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.832151 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.832054    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.832362 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.832336    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.832485 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.832444    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.834273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.834252    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.834613 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.834260    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.845989 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.845959    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.846120 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.846039    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb9qh\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh\") pod \"e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") " pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:07.960581 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:07.960488    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:08.128738 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.128708    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:08.131251 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:26:08.131216    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8207113a_b0d5_4c27_b3c4_396a5d141997.slice/crio-fb369f8cf69242a36eb5aed1627ea88726ac605a2b83d93a14a4dead72ec195d WatchSource:0}: Error finding container fb369f8cf69242a36eb5aed1627ea88726ac605a2b83d93a14a4dead72ec195d: Status 404 returned error can't find the container with id fb369f8cf69242a36eb5aed1627ea88726ac605a2b83d93a14a4dead72ec195d
Jul 07 00:26:08.133398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.133372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:08.133536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.133495    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:08.133611 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.133593    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:08.133659 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.133643    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:08.297946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.297851    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerStarted","Data":"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"}
Jul 07 00:26:08.297946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.297917    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerStarted","Data":"fb369f8cf69242a36eb5aed1627ea88726ac605a2b83d93a14a4dead72ec195d"}
Jul 07 00:26:08.321971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.321918    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podStartSLOduration=1.321896276 podStartE2EDuration="1.321896276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:08.319153578 +0000 UTC m=+1753.644156843" watchObservedRunningTime="2026-07-07 00:26:08.321896276 +0000 UTC m=+1753.646899559"
Jul 07 00:26:08.961406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:08.961368    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:09.965930 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:09.965903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:10.308449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.308367    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:10.309410 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.309389    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:10.546081 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.546036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:10.679760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.679721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:10.774322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.774266    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:10.973554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:10.973451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:11.481038 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:11.481003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:11.580904 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:11.580873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:14.950311 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:14.950252    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:15.612604 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:15.612568    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:15.713529 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:15.713487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:16.139813 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:16.139767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:22.159080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:22.159041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:22.223127 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:22.223080    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:22.593844 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:22.593738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:22.967347 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:22.967314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:23.032904 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.031272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:23.383601 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.383566    2583 generic.go:356] "Generic (PLEG): container finished" podID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerID="3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0" exitCode=0
Jul 07 00:26:23.383601 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.383605    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerDied","Data":"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"}
Jul 07 00:26:23.384029 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.383990    2583 scope.go:117] "RemoveContainer" containerID="3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"
Jul 07 00:26:23.385019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.384985    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:23.385108 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.385080    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:23.385148 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:23.385136    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:24.390524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:24.390490    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerStarted","Data":"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"}
Jul 07 00:26:24.961018 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:24.960973    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:24.962057 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:24.962031    2583 patch_prober.go:28] interesting pod/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 00:26:24.962156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:24.962088    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 00:26:25.309429 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:25.309347    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:25.961263 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:25.961228    2583 patch_prober.go:28] interesting pod/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 00:26:25.961693 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:25.961286    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 00:26:26.961819 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:26.961779    2583 patch_prober.go:28] interesting pod/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 00:26:26.962235 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:26.961836    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 00:26:27.961585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:27.961542    2583 patch_prober.go:28] interesting pod/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 07 00:26:27.961791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:27.961607    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 07 00:26:28.964735 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:28.964703    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:28.965618 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:28.965596    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:29.027332 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.027256    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:29.127249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.127218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:29.266423 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.266336    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:29.360950 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.360918    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:29.414198 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.414164    2583 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" secret="" err="secret \"e2e-mt-9eadf20a-openshift-default-dockercfg-4wphq\" not found"
Jul 07 00:26:29.633591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.633558    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:29.736325 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.736261    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:29.866387 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:29.866347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:30.418953 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:30.418911    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy" containerID="cri-o://6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86" gracePeriod=30
Jul 07 00:26:35.671974 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.671947    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:35.683116 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683048    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683116 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683088    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683116 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683116    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683150    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb9qh\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683195    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683281    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683331    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683370    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket\") pod \"8207113a-b0d5-4c27-b3c4-396a5d141997\" (UID: \"8207113a-b0d5-4c27-b3c4-396a5d141997\") "
Jul 07 00:26:35.683770 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683438    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:35.683770 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683450    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:35.683770 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683680    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.683770 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683701    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.683978 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683768    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:26:35.683978 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.683898    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:35.684337 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.684287    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data" (OuterVolumeSpecName: "istio-data") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:35.685518 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.685481    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:26:35.685622 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.685578    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token" (OuterVolumeSpecName: "istio-token") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:26:35.685622 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.685586    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh" (OuterVolumeSpecName: "kube-api-access-lb9qh") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "kube-api-access-lb9qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:26:35.685894 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.685879    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8207113a-b0d5-4c27-b3c4-396a5d141997" (UID: "8207113a-b0d5-4c27-b3c4-396a5d141997"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:26:35.770086 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.770049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:35.784357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784330    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784356    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb9qh\" (UniqueName: \"kubernetes.io/projected/8207113a-b0d5-4c27-b3c4-396a5d141997-kube-api-access-lb9qh\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784547 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784368    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784547 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784380    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784547 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784389    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8207113a-b0d5-4c27-b3c4-396a5d141997-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784547 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784396    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.784547 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.784404    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8207113a-b0d5-4c27-b3c4-396a5d141997-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:26:35.878598 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.878562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:35.971735 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:35.971650    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:36.135545 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.135512    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:36.450357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.450321    2583 generic.go:356] "Generic (PLEG): container finished" podID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerID="6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86" exitCode=0
Jul 07 00:26:36.450534 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.450361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerDied","Data":"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"}
Jul 07 00:26:36.450534 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.450391    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"
Jul 07 00:26:36.450534 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.450418    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m" event={"ID":"8207113a-b0d5-4c27-b3c4-396a5d141997","Type":"ContainerDied","Data":"fb369f8cf69242a36eb5aed1627ea88726ac605a2b83d93a14a4dead72ec195d"}
Jul 07 00:26:36.450534 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.450451    2583 scope.go:117] "RemoveContainer" containerID="6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"
Jul 07 00:26:36.459595 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.459577    2583 scope.go:117] "RemoveContainer" containerID="3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"
Jul 07 00:26:36.467897 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.467877    2583 scope.go:117] "RemoveContainer" containerID="6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"
Jul 07 00:26:36.468253 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:26:36.468234    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86\": container with ID starting with 6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86 not found: ID does not exist" containerID="6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"
Jul 07 00:26:36.468340 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.468262    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86"} err="failed to get container status \"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86\": rpc error: code = NotFound desc = could not find container \"6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86\": container with ID starting with 6456b3249af9ca8d3ba065a020817c2ca14746adbb3aa8747930ceda1f390f86 not found: ID does not exist"
Jul 07 00:26:36.468340 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.468279    2583 scope.go:117] "RemoveContainer" containerID="3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"
Jul 07 00:26:36.468552 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:26:36.468533    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0\": container with ID starting with 3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0 not found: ID does not exist" containerID="3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"
Jul 07 00:26:36.468595 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.468559    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0"} err="failed to get container status \"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0\": rpc error: code = NotFound desc = could not find container \"3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0\": container with ID starting with 3334063957f256d16163f7f2529d2f87408a12dbd4c752ed25f09b9f03e57ea0 not found: ID does not exist"
Jul 07 00:26:36.480677 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.480636    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:36.484999 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:36.484966    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9eadf20a-openshift-default-5d89fc54d7-2j96m"]
Jul 07 00:26:36.488898 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:26:36.488863    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8207113a_b0d5_4c27_b3c4_396a5d141997.slice\": RecentStats: unable to find data in memory cache]"
Jul 07 00:26:37.158971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:37.158926    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" path="/var/lib/kubelet/pods/8207113a-b0d5-4c27-b3c4-396a5d141997/volumes"
Jul 07 00:26:41.798399 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:41.798357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:41.899525 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:41.899485    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:48.691188 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.691094    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:48.985837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.985745    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:26:48.986360 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986333    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.986360 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986358    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.986606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986375    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.986606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986384    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.986606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986499    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.986606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.986514    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8207113a-b0d5-4c27-b3c4-396a5d141997" containerName="istio-proxy"
Jul 07 00:26:48.989372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.989349    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:48.992165 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:48.992148    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-efed8c96-openshift-default-dockercfg-shvzd\""
Jul 07 00:26:49.003284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.003256    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:26:49.010416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.010387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:49.103737 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103737 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103738    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103767    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103816    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103849    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103948    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r574s\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.103988 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.103981    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.104205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.104016    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.104205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.104040    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205497    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205541    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r574s\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.205732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205695    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205737    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205760    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206016 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.205790    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206181 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.206127    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206181 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.206140    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206288 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.206202    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206456 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.206435    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.206621 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.206598    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.207878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.207860    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.208142 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.208125    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.221403 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.221376    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.230111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.230089    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r574s\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s\") pod \"e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") " pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.243543 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.243476    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:49.299678 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.299641    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:49.440263 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.440225    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:26:49.441429 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:26:49.441396    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b7a00e4_60c1_4e66_ab27_0174da030208.slice/crio-ab44dbfe07dd04239a82e36587b22b5870fcad8e0690c67a5bd54f880632177a WatchSource:0}: Error finding container ab44dbfe07dd04239a82e36587b22b5870fcad8e0690c67a5bd54f880632177a: Status 404 returned error can't find the container with id ab44dbfe07dd04239a82e36587b22b5870fcad8e0690c67a5bd54f880632177a
Jul 07 00:26:49.443449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.443418    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:49.443537 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.443482    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:49.443537 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.443512    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:26:49.519070 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.519023    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerStarted","Data":"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"}
Jul 07 00:26:49.519238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.519095    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerStarted","Data":"ab44dbfe07dd04239a82e36587b22b5870fcad8e0690c67a5bd54f880632177a"}
Jul 07 00:26:49.542476 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:49.542423    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podStartSLOduration=1.542404879 podStartE2EDuration="1.542404879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:26:49.539468537 +0000 UTC m=+1794.864471833" watchObservedRunningTime="2026-07-07 00:26:49.542404879 +0000 UTC m=+1794.867408225"
Jul 07 00:26:50.300405 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:50.300363    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:50.305754 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:50.305725    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:50.530833 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:50.530796    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:50.531774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:50.531748    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:26:51.495131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:51.495096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:51.590395 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:51.590347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:51.676102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:51.676058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:51.830345 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:51.830222    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:52.139124 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:52.139082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:52.245184 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:52.245152    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:26:55.275832 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:55.275803    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:26:55.283471 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:55.283442    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:26:55.993502 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:26:55.993470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:02.314891 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:02.314857    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:02.416488 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:02.416453    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:05.850757 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:05.850726    2583 patch_prober.go:28] interesting pod/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:33342->10.132.0.52:15021: read: connection reset by peer" start-of-body=
Jul 07 00:27:05.851110 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:05.850836    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": read tcp 10.132.0.2:33342->10.132.0.52:15021: read: connection reset by peer"
Jul 07 00:27:06.612628 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.612593    2583 generic.go:356] "Generic (PLEG): container finished" podID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerID="dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905" exitCode=0
Jul 07 00:27:06.612798 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.612669    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerDied","Data":"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"}
Jul 07 00:27:06.613237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.613221    2583 scope.go:117] "RemoveContainer" containerID="dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"
Jul 07 00:27:06.614277 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.614250    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:27:06.614363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.614349    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:27:06.614404 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:06.614396    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:27:07.623774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:07.623732    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerStarted","Data":"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"}
Jul 07 00:27:08.299892 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:08.299854    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:27:08.300117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:08.300094    2583 patch_prober.go:28] interesting pod/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 07 00:27:08.300165 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:08.300144    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 07 00:27:08.663346 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:08.663315    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:08.716942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:08.716908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:09.300445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:09.300403    2583 patch_prober.go:28] interesting pod/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 07 00:27:09.300657 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:09.300461    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 07 00:27:10.300528 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:10.300491    2583 patch_prober.go:28] interesting pod/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 07 00:27:10.300946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:10.300550    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 07 00:27:11.300974 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:11.300937    2583 patch_prober.go:28] interesting pod/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 07 00:27:11.301419 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:11.300998    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 07 00:27:12.303646 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:12.303615    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:27:12.304122 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:12.303887    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:27:12.304679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:12.304661    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:27:12.879582 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:12.879504    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:12.985307 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:12.985190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:14.898942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.898908    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:27:14.902011 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.901988    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.905265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.905239    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-efed8c96\"/\"e2e-lifecycle-model-efed8c96-kserve-self-signed-certs\""
Jul 07 00:27:14.906312 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.906270    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-efed8c96\"/\"kube-root-ca.crt\""
Jul 07 00:27:14.906427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.906308    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-efed8c96\"/\"default-dockercfg-6rw8x\""
Jul 07 00:27:14.906427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.906359    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-efed8c96\"/\"openshift-service-ca.crt\""
Jul 07 00:27:14.915208 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.915177    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:27:14.945018 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.944978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:14.953191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.953363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953210    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.953363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlhm\" (UniqueName: \"kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.953363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953310    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.953533 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953379    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:14.953533 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:14.953441    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054421 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054378    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054445    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054540    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054571    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054604    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlhm\" (UniqueName: \"kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054843    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054943 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054883    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.054943 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.054921    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.056887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.056865    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.057058 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.057043    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.064159 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.064132    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlhm\" (UniqueName: \"kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm\") pod \"e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") " pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.212714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.212598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:15.357713 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.357671    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:27:15.357713 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:27:15.357688    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod485613d7_f711_45af_9693_634b1d5eb3d5.slice/crio-3b970c9f99a363654a67676806b4c4bc456c33559bfc4ed958e4de0c85543a45 WatchSource:0}: Error finding container 3b970c9f99a363654a67676806b4c4bc456c33559bfc4ed958e4de0c85543a45: Status 404 returned error can't find the container with id 3b970c9f99a363654a67676806b4c4bc456c33559bfc4ed958e4de0c85543a45
Jul 07 00:27:15.678979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.678936    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" event={"ID":"485613d7-f711-45af-9693-634b1d5eb3d5","Type":"ContainerStarted","Data":"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"}
Jul 07 00:27:15.678979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.678977    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" event={"ID":"485613d7-f711-45af-9693-634b1d5eb3d5","Type":"ContainerStarted","Data":"3b970c9f99a363654a67676806b4c4bc456c33559bfc4ed958e4de0c85543a45"}
Jul 07 00:27:15.699371 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:15.699312    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" podStartSLOduration=1.6992813249999998 podStartE2EDuration="1.699281325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:15.696710481 +0000 UTC m=+1821.021713766" watchObservedRunningTime="2026-07-07 00:27:15.699281325 +0000 UTC m=+1821.024284608"
Jul 07 00:27:25.213147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:25.213108    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:25.213609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:25.213246    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:25.226255 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:25.226228    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:25.744514 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:25.744483    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:27:38.754156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:38.754119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:39.112248 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:39.112161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:44.873187 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:44.873155    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:27:49.853183 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:49.853142    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:27:49.947894 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:27:49.947850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:04.875882 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:04.875830    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" podUID="485613d7-f711-45af-9693-634b1d5eb3d5" containerName="main" containerID="cri-o://99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd" gracePeriod=45
Jul 07 00:28:05.136198 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.136129    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:28:05.305205 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305174    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305236    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jlhm\" (UniqueName: \"kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305266    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305284    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305369    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305399    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs\") pod \"485613d7-f711-45af-9693-634b1d5eb3d5\" (UID: \"485613d7-f711-45af-9693-634b1d5eb3d5\") "
Jul 07 00:28:05.305732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305535    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home" (OuterVolumeSpecName: "home") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:05.305732 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305676    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:05.305876 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305743    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-home\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.305930 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.305895    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache" (OuterVolumeSpecName: "model-cache") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:05.307667 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.307634    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm" (OuterVolumeSpecName: "kube-api-access-4jlhm") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "kube-api-access-4jlhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:28:05.307667 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.307662    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm" (OuterVolumeSpecName: "dshm") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:05.307883 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.307677    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "485613d7-f711-45af-9693-634b1d5eb3d5" (UID: "485613d7-f711-45af-9693-634b1d5eb3d5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:28:05.406932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.406833    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jlhm\" (UniqueName: \"kubernetes.io/projected/485613d7-f711-45af-9693-634b1d5eb3d5-kube-api-access-4jlhm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.406932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.406870    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-model-cache\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.406932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.406882    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-dshm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.406932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.406894    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/485613d7-f711-45af-9693-634b1d5eb3d5-tmp-dir\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.406932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.406904    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/485613d7-f711-45af-9693-634b1d5eb3d5-tls-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:05.940280 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.940242    2583 generic.go:356] "Generic (PLEG): container finished" podID="485613d7-f711-45af-9693-634b1d5eb3d5" containerID="99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd" exitCode=0
Jul 07 00:28:05.940280 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.940305    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"
Jul 07 00:28:05.940858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.940328    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" event={"ID":"485613d7-f711-45af-9693-634b1d5eb3d5","Type":"ContainerDied","Data":"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"}
Jul 07 00:28:05.940858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.940373    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b" event={"ID":"485613d7-f711-45af-9693-634b1d5eb3d5","Type":"ContainerDied","Data":"3b970c9f99a363654a67676806b4c4bc456c33559bfc4ed958e4de0c85543a45"}
Jul 07 00:28:05.940858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.940391    2583 scope.go:117] "RemoveContainer" containerID="99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"
Jul 07 00:28:05.949558 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.949538    2583 scope.go:117] "RemoveContainer" containerID="99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"
Jul 07 00:28:05.949818 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:28:05.949800    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd\": container with ID starting with 99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd not found: ID does not exist" containerID="99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"
Jul 07 00:28:05.949861 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.949829    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd"} err="failed to get container status \"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd\": rpc error: code = NotFound desc = could not find container \"99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd\": container with ID starting with 99d253842d34c74f1b951ce0c3a8958a252877a0c298d763e4d7abf964ed88bd not found: ID does not exist"
Jul 07 00:28:05.962484 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.962458    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:28:05.966620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:05.966597    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-efed8c96/e2e-lifecycle-model-efed8c96-kserve-7b568dbc47-xsl6b"]
Jul 07 00:28:07.158725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:07.158691    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="485613d7-f711-45af-9693-634b1d5eb3d5" path="/var/lib/kubelet/pods/485613d7-f711-45af-9693-634b1d5eb3d5/volumes"
Jul 07 00:28:21.849704 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:21.849670    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:21.973351 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:21.973317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:27.543985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:27.543954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:33.460488 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:33.460446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:33.549607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:33.549555    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:33.698347 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:33.698310    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:28:33.698569 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:33.698537    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy" containerID="cri-o://cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895" gracePeriod=30
Jul 07 00:28:33.729029 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:33.728936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:35.028408 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:35.028352    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:36.195237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:36.195200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:37.749345 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:37.749286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:38.966456 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:38.966369    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:28:39.114448 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.114412    2583 generic.go:356] "Generic (PLEG): container finished" podID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerID="cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895" exitCode=0
Jul 07 00:28:39.114620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.114458    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"
Jul 07 00:28:39.114620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.114493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerDied","Data":"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"}
Jul 07 00:28:39.114620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.114553    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9" event={"ID":"4b7a00e4-60c1-4e66-ab27-0174da030208","Type":"ContainerDied","Data":"ab44dbfe07dd04239a82e36587b22b5870fcad8e0690c67a5bd54f880632177a"}
Jul 07 00:28:39.114620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.114579    2583 scope.go:117] "RemoveContainer" containerID="cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"
Jul 07 00:28:39.116811 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116792    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116820    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116838    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116858    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116883    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116918    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.116944 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116945    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.117278 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.116975    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r574s\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.117278 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117026    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy\") pod \"4b7a00e4-60c1-4e66-ab27-0174da030208\" (UID: \"4b7a00e4-60c1-4e66-ab27-0174da030208\") "
Jul 07 00:28:39.117278 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117061    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:39.117278 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117120    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:39.117501 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117382    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:39.117501 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117399    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.117501 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117420    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.117656 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117573    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:28:39.117844 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.117785    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:39.119575 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.119549    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:28:39.119668 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.119610    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s" (OuterVolumeSpecName: "kube-api-access-r574s") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "kube-api-access-r574s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:28:39.119668 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.119615    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:28:39.119884 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.119870    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b7a00e4-60c1-4e66-ab27-0174da030208" (UID: "4b7a00e4-60c1-4e66-ab27-0174da030208"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:28:39.138526 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.138503    2583 scope.go:117] "RemoveContainer" containerID="dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"
Jul 07 00:28:39.147057 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.147034    2583 scope.go:117] "RemoveContainer" containerID="cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"
Jul 07 00:28:39.147361 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:28:39.147335    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895\": container with ID starting with cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895 not found: ID does not exist" containerID="cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"
Jul 07 00:28:39.147452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.147369    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895"} err="failed to get container status \"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895\": rpc error: code = NotFound desc = could not find container \"cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895\": container with ID starting with cb205ccdd1bd3e238934367fb913f62aa3a89ea63d3402ca1e99a7af58f9b895 not found: ID does not exist"
Jul 07 00:28:39.147452 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.147387    2583 scope.go:117] "RemoveContainer" containerID="dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"
Jul 07 00:28:39.147643 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:28:39.147629    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905\": container with ID starting with dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905 not found: ID does not exist" containerID="dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"
Jul 07 00:28:39.147683 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.147647    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905"} err="failed to get container status \"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905\": rpc error: code = NotFound desc = could not find container \"dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905\": container with ID starting with dc52f0dc1feedd5970e66fada689289f3ccfa9e5664107bac3e1c45e4920c905 not found: ID does not exist"
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217845    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b7a00e4-60c1-4e66-ab27-0174da030208-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217878    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217886    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217894    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217901    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217911    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r574s\" (UniqueName: \"kubernetes.io/projected/4b7a00e4-60c1-4e66-ab27-0174da030208-kube-api-access-r574s\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.217932 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.217919    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b7a00e4-60c1-4e66-ab27-0174da030208-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:28:39.431517 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.431486    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:28:39.437421 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:39.437396    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-efed8c96-openshift-default-85f84b6c9f-b6fb9"]
Jul 07 00:28:41.158194 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:41.158158    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" path="/var/lib/kubelet/pods/4b7a00e4-60c1-4e66-ab27-0174da030208/volumes"
Jul 07 00:28:43.768917 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:43.768869    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:43.869933 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:43.869902    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:44.537664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:44.537629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:50.534374 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:50.534340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:50.602133 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:50.602100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:50.693052 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:50.693017    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:28:50.794058 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:28:50.793974    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:01.286924 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:01.286885    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:01.643337 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:01.643278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:04.349356 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:04.349326    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:22.653891 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:22.653854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:28.558769 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:28.558733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:28.667359 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:28.667323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:28.930172 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:28.930140    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:29.257770 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.257683    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:29:29.258077 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258063    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.258131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258079    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.258131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258092    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.258131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258097    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.258131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258107    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="485613d7-f711-45af-9693-634b1d5eb3d5" containerName="main"
Jul 07 00:29:29.258131 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258113    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="485613d7-f711-45af-9693-634b1d5eb3d5" containerName="main"
Jul 07 00:29:29.258317 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258171    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="485613d7-f711-45af-9693-634b1d5eb3d5" containerName="main"
Jul 07 00:29:29.258317 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258181    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.258317 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.258187    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b7a00e4-60c1-4e66-ab27-0174da030208" containerName="istio-proxy"
Jul 07 00:29:29.260864 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.260847    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.263584 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.263561    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e9266c-openshift-default-dockercfg-d9zv6\""
Jul 07 00:29:29.273998 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.273971    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:29:29.283836 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.283809    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:29.391401 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391355    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391599 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391599 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391467    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391599 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391599 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391565    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391775 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391633    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mcb4\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391775 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391775 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391713    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.391775 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.391732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.492965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.492917    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.492965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.492970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493089    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mcb4\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493197    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493590 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493220    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493590 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493521    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493590 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493544    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493665    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493875    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.493979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.493901    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.495239 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.495223    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.495671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.495653    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.501786 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.501763    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.501899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.501883    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mcb4\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4\") pod \"e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") " pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.503834 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.503810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:29.571715 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.571623    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:29.716108 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.716079    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:29:29.717464 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:29:29.717435    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845e1939_9ee9_42d7_b97e_fdfbf1fd62cc.slice/crio-6d1926f5bd8147af560c76bfdeeb4246a69368c11af1cb5a050d50943d69625e WatchSource:0}: Error finding container 6d1926f5bd8147af560c76bfdeeb4246a69368c11af1cb5a050d50943d69625e: Status 404 returned error can't find the container with id 6d1926f5bd8147af560c76bfdeeb4246a69368c11af1cb5a050d50943d69625e
Jul 07 00:29:29.720111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.720080    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:29.720178 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.720146    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:29.720217 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:29.720179    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:30.356030 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:30.355992    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerStarted","Data":"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"}
Jul 07 00:29:30.356254 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:30.356052    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerStarted","Data":"6d1926f5bd8147af560c76bfdeeb4246a69368c11af1cb5a050d50943d69625e"}
Jul 07 00:29:30.383227 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:30.383167    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podStartSLOduration=1.383141812 podStartE2EDuration="1.383141812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:30.381243162 +0000 UTC m=+1955.706246448" watchObservedRunningTime="2026-07-07 00:29:30.383141812 +0000 UTC m=+1955.708145095"
Jul 07 00:29:30.572407 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:30.572370    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:30.577261 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:30.577236    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:31.360780 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:31.360749    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:31.361618 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:31.361598    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:32.361557 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:32.361524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:32.451226 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:32.451189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:32.591247 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:32.591208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:33.178646 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:33.178601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:33.280103 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:33.280048    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:36.889595 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:36.889561    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:37.252670 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.252562    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:37.535213 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.535095    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:29:37.539504 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.539477    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.542062 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.542029    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-dd30cd-openshift-default-dockercfg-2htmf\""
Jul 07 00:29:37.548675 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.548651    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:29:37.562554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562524    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.562774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562734    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.562867 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562844    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.562911 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562882    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.562955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.562992 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.562967    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.563035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.563002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.563035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.563026    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.563110 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.563051    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28mm2\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.566662 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.566632    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:37.663520 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663730 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663530    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663730 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663556    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663730 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663610    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mm2\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663730 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663746    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663772    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.663957 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663920    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.664233 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.663995    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.664447 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.664422    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.664447 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.664437    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.664584 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.664516    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.666712 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.666686    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.666849 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.666737    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.667150 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.667128    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:37.674034 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.674007    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.674149 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.674034    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mm2\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2\") pod \"e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") " pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:37.852229 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:37.852129    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:38.009968 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.009941    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:29:38.012102 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:29:38.012075    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod111ddc50_80ee_4631_a699_540aff53f12e.slice/crio-4da6b9279b2a27e88ca685b5531f0c57897f44b61229b319fcd5147a910e70f6 WatchSource:0}: Error finding container 4da6b9279b2a27e88ca685b5531f0c57897f44b61229b319fcd5147a910e70f6: Status 404 returned error can't find the container with id 4da6b9279b2a27e88ca685b5531f0c57897f44b61229b319fcd5147a910e70f6
Jul 07 00:29:38.014215 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.014177    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:38.014308 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.014255    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:38.014308 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.014282    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:38.393955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.393915    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerStarted","Data":"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"}
Jul 07 00:29:38.394152 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.393969    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerStarted","Data":"4da6b9279b2a27e88ca685b5531f0c57897f44b61229b319fcd5147a910e70f6"}
Jul 07 00:29:38.414162 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.414105    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podStartSLOduration=1.4140870269999999 podStartE2EDuration="1.414087027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:29:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:29:38.412388453 +0000 UTC m=+1963.737391737" watchObservedRunningTime="2026-07-07 00:29:38.414087027 +0000 UTC m=+1963.739090310"
Jul 07 00:29:38.852357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:38.852249    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:39.857924 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:39.857892    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:40.355128 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.355093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:40.403223 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.403191    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:40.404162 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.404140    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:40.460282 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.460229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:40.525440 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.525404    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:40.878937 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:40.878904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:41.101113 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:41.101078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:41.202069 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:41.201976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:44.941779 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:44.941738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:47.402627 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:47.402519    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 00:29:47.402627 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:47.402603    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:29:48.444613 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.444579    2583 generic.go:356] "Generic (PLEG): container finished" podID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerID="95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9" exitCode=0
Jul 07 00:29:48.445066 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.444651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerDied","Data":"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"}
Jul 07 00:29:48.445066 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.445055    2583 scope.go:117] "RemoveContainer" containerID="95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"
Jul 07 00:29:48.446038 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.446009    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:48.446158 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.446115    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:48.446223 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.446171    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:48.541372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:48.541338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:49.456059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:49.455093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerStarted","Data":"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"}
Jul 07 00:29:49.571853 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:49.571812    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:49.572204 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:49.572174    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 07 00:29:49.572459 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:49.572415    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 07 00:29:50.573024 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:50.572983    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 07 00:29:50.573757 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:50.573712    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 07 00:29:51.572493 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:51.572441    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 07 00:29:51.572722 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:51.572518    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 07 00:29:52.572540 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:52.572499    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 07 00:29:52.573140 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:52.573108    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 07 00:29:53.576285 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:53.576247    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:53.576761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:53.576638    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:53.577339 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:53.577320    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:29:54.027409 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:54.027376    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:54.128789 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:54.128749    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:29:55.577653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:55.576973    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:47370->10.132.0.55:15021: read: connection reset by peer" start-of-body=
Jul 07 00:29:55.577653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:55.577035    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": read tcp 10.132.0.2:47370->10.132.0.55:15021: read: connection reset by peer"
Jul 07 00:29:56.502372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.502336    2583 generic.go:356] "Generic (PLEG): container finished" podID="111ddc50-80ee-4631-a699-540aff53f12e" containerID="af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7" exitCode=0
Jul 07 00:29:56.502554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.502408    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerDied","Data":"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"}
Jul 07 00:29:56.502896 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.502875    2583 scope.go:117] "RemoveContainer" containerID="af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"
Jul 07 00:29:56.503946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.503915    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:56.504007 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.503989    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:56.504047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:56.504022    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:29:57.509256 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:57.509216    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerStarted","Data":"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"}
Jul 07 00:29:57.852445 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:57.852337    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:29:57.852756 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:57.852731    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 00:29:57.852933 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:57.852905    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 00:29:58.852632 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:58.852587    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 00:29:58.853179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:58.852663    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 00:29:59.853151 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:59.853110    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 00:29:59.853677 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:29:59.853389    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 00:30:00.149936 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.149885    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:30:00.154322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.154274    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:30:00.158242 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.158208    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 07 00:30:00.158424 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.158326    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lq9jt\""
Jul 07 00:30:00.159245 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.159224    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 07 00:30:00.174551 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.174488    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:30:00.288434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.288393    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhqbh\" (UniqueName: \"kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh\") pod \"maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7\" (UID: \"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:30:00.388971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.388927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhqbh\" (UniqueName: \"kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh\") pod \"maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7\" (UID: \"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:30:00.398181 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.398152    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhqbh\" (UniqueName: \"kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh\") pod \"maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7\" (UID: \"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:30:00.467117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.467014    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:30:00.646373 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.640897    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:30:00.661869 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:30:00.661818    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a77db02_a1c4_4f38_9dd2_8104e2a5b2b8.slice/crio-9c96603d2c9370969d00510dfe31544fc3b6cae51ad5d7d23e879c50a0b26be3 WatchSource:0}: Error finding container 9c96603d2c9370969d00510dfe31544fc3b6cae51ad5d7d23e879c50a0b26be3: Status 404 returned error can't find the container with id 9c96603d2c9370969d00510dfe31544fc3b6cae51ad5d7d23e879c50a0b26be3
Jul 07 00:30:00.854053 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.853543    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 07 00:30:00.854053 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:00.853607    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 07 00:30:01.543939 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.543900    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" event={"ID":"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8","Type":"ContainerStarted","Data":"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"}
Jul 07 00:30:01.544118 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.543959    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" event={"ID":"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8","Type":"ContainerStarted","Data":"9c96603d2c9370969d00510dfe31544fc3b6cae51ad5d7d23e879c50a0b26be3"}
Jul 07 00:30:01.560120 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.560064    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" podStartSLOduration=1.560046233 podStartE2EDuration="1.560046233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:01.558903569 +0000 UTC m=+1986.883906857" watchObservedRunningTime="2026-07-07 00:30:01.560046233 +0000 UTC m=+1986.885049516"
Jul 07 00:30:01.815178 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.815078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:01.856042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.856010    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:30:01.856475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.856318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:30:01.857116 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.857098    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:30:01.880042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:01.880012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:14.587835 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.587793    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:30:14.592012 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.591993    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.594725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.594702    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e9266c\"/\"kube-root-ca.crt\""
Jul 07 00:30:14.595791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.595776    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e9266c\"/\"test-model-e9266c-kserve-self-signed-certs\""
Jul 07 00:30:14.595874 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.595799    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-e9266c\"/\"openshift-service-ca.crt\""
Jul 07 00:30:14.595874 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.595809    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-e9266c\"/\"default-dockercfg-x5n5b\""
Jul 07 00:30:14.600333 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.600315    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:30:14.613992 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.613960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:30:14.718473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718431    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.718687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718493    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.718687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718533    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-574mz\" (UniqueName: \"kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.718687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718561    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.718687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.718687 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.718671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819739 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819701    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819752    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-574mz\" (UniqueName: \"kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819832    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.819928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.819850    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.820240 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.820217    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.820333 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.820254    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.820383 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.820343    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.822070 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.822047    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.822275 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.822255    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.828575 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.828553    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-574mz\" (UniqueName: \"kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz\") pod \"test-model-e9266c-kserve-76cddcff5f-b62mm\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") " pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:14.902034 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:14.901998    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:15.034829 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:15.034805    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:30:15.037022 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:30:15.036992    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeb7bbe3d_0a36_4fc4_b520_0ec2cb0c10e0.slice/crio-35030c417ae9276e8534bb9d03181c7f014108b924f56ace2e0ff410e97e3eef WatchSource:0}: Error finding container 35030c417ae9276e8534bb9d03181c7f014108b924f56ace2e0ff410e97e3eef: Status 404 returned error can't find the container with id 35030c417ae9276e8534bb9d03181c7f014108b924f56ace2e0ff410e97e3eef
Jul 07 00:30:15.614204 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:15.614164    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" event={"ID":"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0","Type":"ContainerStarted","Data":"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"}
Jul 07 00:30:15.614718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:15.614215    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" event={"ID":"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0","Type":"ContainerStarted","Data":"35030c417ae9276e8534bb9d03181c7f014108b924f56ace2e0ff410e97e3eef"}
Jul 07 00:30:24.902335 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:24.902286    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:24.902868 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:24.902354    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:24.914864 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:24.914837    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:24.931614 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:24.931560    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" podStartSLOduration=10.931543698 podStartE2EDuration="10.931543698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:30:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:30:15.631314946 +0000 UTC m=+2000.956318232" watchObservedRunningTime="2026-07-07 00:30:24.931543698 +0000 UTC m=+2010.256546983"
Jul 07 00:30:25.678060 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:25.678020    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:30:49.431674 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:30:49.431640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:00.010466 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:00.010423    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:31:00.011111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:00.010701    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" podUID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" containerName="cleanup" containerID="cri-o://ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85" gracePeriod=30
Jul 07 00:31:09.467526 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:09.467493    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:31:09.489071 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:09.489040    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:10.414612 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:10.414580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:10.587124 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:10.587085    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:11.005246 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:11.005210    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:11.105492 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:11.105448    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:11.272521 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:11.272427    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:14.631476 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:14.631446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:14.732888 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:14.732855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:16.079620 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:16.079586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:24.488418 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.488370    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" podUID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" containerName="main" containerID="cri-o://00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c" gracePeriod=45
Jul 07 00:31:24.745817 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.745739    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:31:24.801348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801309    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-574mz\" (UniqueName: \"kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801575    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801644 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801627    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801683    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801754 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs\") pod \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\" (UID: \"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0\") "
Jul 07 00:31:24.801826 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801803    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home" (OuterVolumeSpecName: "home") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:24.801965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801896    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:24.802059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.801994    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:24.802113 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.802085    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tmp-dir\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.802113 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.802099    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-home\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.802113 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.802107    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-model-cache\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.803730 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.803705    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:31:24.803840 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.803700    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm" (OuterVolumeSpecName: "dshm") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:24.803840 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.803760    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz" (OuterVolumeSpecName: "kube-api-access-574mz") pod "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" (UID: "eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0"). InnerVolumeSpecName "kube-api-access-574mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:24.903111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.903081    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-dshm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.903111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.903106    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-tls-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.903111 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.903117    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-574mz\" (UniqueName: \"kubernetes.io/projected/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0-kube-api-access-574mz\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:24.978237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.978194    2583 generic.go:356] "Generic (PLEG): container finished" podID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" containerID="00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c" exitCode=0
Jul 07 00:31:24.978237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.978239    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"
Jul 07 00:31:24.978511 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.978287    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" event={"ID":"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0","Type":"ContainerDied","Data":"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"}
Jul 07 00:31:24.978511 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.978357    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm" event={"ID":"eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0","Type":"ContainerDied","Data":"35030c417ae9276e8534bb9d03181c7f014108b924f56ace2e0ff410e97e3eef"}
Jul 07 00:31:24.978511 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.978377    2583 scope.go:117] "RemoveContainer" containerID="00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"
Jul 07 00:31:24.988061 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.988044    2583 scope.go:117] "RemoveContainer" containerID="00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"
Jul 07 00:31:24.988349 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:31:24.988332    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c\": container with ID starting with 00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c not found: ID does not exist" containerID="00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"
Jul 07 00:31:24.988397 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.988357    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c"} err="failed to get container status \"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c\": rpc error: code = NotFound desc = could not find container \"00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c\": container with ID starting with 00ef19fda868f7476bf5f1781f97ed75792878156e3a5039aba488228e8f995c not found: ID does not exist"
Jul 07 00:31:24.999739 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:24.999683    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:31:25.003702 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:25.003678    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-e9266c/test-model-e9266c-kserve-76cddcff5f-b62mm"]
Jul 07 00:31:25.158811 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:25.158782    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" path="/var/lib/kubelet/pods/eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0/volumes"
Jul 07 00:31:30.191804 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:30.191780    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:31:30.252800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:30.252762    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhqbh\" (UniqueName: \"kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh\") pod \"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8\" (UID: \"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8\") "
Jul 07 00:31:30.254828 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:30.254802    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh" (OuterVolumeSpecName: "kube-api-access-mhqbh") pod "5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" (UID: "5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8"). InnerVolumeSpecName "kube-api-access-mhqbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:30.354594 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:30.354502    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhqbh\" (UniqueName: \"kubernetes.io/projected/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8-kube-api-access-mhqbh\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:31.010283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.010245    2583 generic.go:356] "Generic (PLEG): container finished" podID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" containerID="ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85" exitCode=137
Jul 07 00:31:31.010483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.010315    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"
Jul 07 00:31:31.010483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.010327    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" event={"ID":"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8","Type":"ContainerDied","Data":"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"}
Jul 07 00:31:31.010483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.010389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7" event={"ID":"5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8","Type":"ContainerDied","Data":"9c96603d2c9370969d00510dfe31544fc3b6cae51ad5d7d23e879c50a0b26be3"}
Jul 07 00:31:31.010483 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.010413    2583 scope.go:117] "RemoveContainer" containerID="ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"
Jul 07 00:31:31.019671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.019464    2583 scope.go:117] "RemoveContainer" containerID="ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"
Jul 07 00:31:31.019748 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:31:31.019730    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85\": container with ID starting with ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85 not found: ID does not exist" containerID="ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"
Jul 07 00:31:31.019791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.019757    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85"} err="failed to get container status \"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85\": rpc error: code = NotFound desc = could not find container \"ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85\": container with ID starting with ae248504ceec6efd5ae3552148174aae8d521fc3fd190a9f481d2422961e6d85 not found: ID does not exist"
Jul 07 00:31:31.031539 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.031515    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:31:31.035742 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.035721    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-e9266c-29723070-nd7m7"]
Jul 07 00:31:31.158119 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:31.158084    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" path="/var/lib/kubelet/pods/5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8/volumes"
Jul 07 00:31:52.952898 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:52.952856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:53.230561 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.230482    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:31:53.230751 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.230703    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" containerID="cri-o://552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65" gracePeriod=30
Jul 07 00:31:53.257375 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.257336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:53.579071 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.578982    2583 patch_prober.go:28] interesting pod/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:31:53.579232 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.579066    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:31:53.728410 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:53.728372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:54.217805 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:54.217770    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:54.325492 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:54.325441    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:54.460147 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:54.460113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:54.564649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:54.564569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:31:55.306152 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:55.306122    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:31:55.315362 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:55.315339    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:31:58.485136 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.485110    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:31:58.522344 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522245    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522344 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522319    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522488    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mcb4\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522594 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522575    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522632    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522688    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522767 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522717    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522816 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522763    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.522864 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.522826    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert\") pod \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\" (UID: \"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc\") "
Jul 07 00:31:58.523504 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.523211    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data" (OuterVolumeSpecName: "istio-data") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:58.523504 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.523359    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:58.523504 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.523462    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:31:58.523504 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.523450    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:58.523723 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.523518    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:58.525059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.525029    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:31:58.525059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.525045    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token" (OuterVolumeSpecName: "istio-token") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:58.525059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.525036    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4" (OuterVolumeSpecName: "kube-api-access-7mcb4") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "kube-api-access-7mcb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:31:58.525322 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.525092    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" (UID: "845e1939-9ee9-42d7-b97e-fdfbf1fd62cc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:31:58.623901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623860    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.623901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623893    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mcb4\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-kube-api-access-7mcb4\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.623901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623904    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.623901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623914    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.624168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623922    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.624168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623930    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.624168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623940    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.624168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623947    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:58.624168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:58.623955    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:31:59.156533 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.156497    2583 generic.go:356] "Generic (PLEG): container finished" podID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerID="552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65" exitCode=0
Jul 07 00:31:59.156716 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.156600    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"
Jul 07 00:31:59.158616 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.158587    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerDied","Data":"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"}
Jul 07 00:31:59.158703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.158645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g" event={"ID":"845e1939-9ee9-42d7-b97e-fdfbf1fd62cc","Type":"ContainerDied","Data":"6d1926f5bd8147af560c76bfdeeb4246a69368c11af1cb5a050d50943d69625e"}
Jul 07 00:31:59.158703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.158681    2583 scope.go:117] "RemoveContainer" containerID="552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"
Jul 07 00:31:59.168772 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.168752    2583 scope.go:117] "RemoveContainer" containerID="95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"
Jul 07 00:31:59.176597 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.176580    2583 scope.go:117] "RemoveContainer" containerID="552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"
Jul 07 00:31:59.176819 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:31:59.176799    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65\": container with ID starting with 552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65 not found: ID does not exist" containerID="552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"
Jul 07 00:31:59.176903 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.176831    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65"} err="failed to get container status \"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65\": rpc error: code = NotFound desc = could not find container \"552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65\": container with ID starting with 552bfef8215676549e6c1b12f2fa524031a7582d28f9073b476ef6884e4fda65 not found: ID does not exist"
Jul 07 00:31:59.176903 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.176854    2583 scope.go:117] "RemoveContainer" containerID="95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"
Jul 07 00:31:59.177095 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:31:59.177074    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9\": container with ID starting with 95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9 not found: ID does not exist" containerID="95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"
Jul 07 00:31:59.177137 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.177102    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9"} err="failed to get container status \"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9\": rpc error: code = NotFound desc = could not find container \"95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9\": container with ID starting with 95a74ac5ba24f298a3448ac7bf8238cc697bdd4fcf4f07935812cbe63eea6cd9 not found: ID does not exist"
Jul 07 00:31:59.182649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.182629    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:31:59.186302 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:31:59.186266    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e9266c-openshift-default-5ffb5cb879-plt2g"]
Jul 07 00:32:00.011904 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.011870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:00.079011 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.078982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:00.282133 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.282050    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:32:00.282348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.282322    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" containerID="cri-o://a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226" gracePeriod=30
Jul 07 00:32:00.306883 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.306855    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:00.931037 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:00.931002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:01.159150 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.159110    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" path="/var/lib/kubelet/pods/845e1939-9ee9-42d7-b97e-fdfbf1fd62cc/volumes"
Jul 07 00:32:01.226673 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.226598    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:01.226992 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.226978    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" containerName="main"
Jul 07 00:32:01.226992 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.226993    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" containerName="main"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227011    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" containerName="cleanup"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227017    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" containerName="cleanup"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227025    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227031    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227038    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:01.227094 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227043    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:01.227338 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227098    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:01.227338 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227105    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb7bbe3d-0a36-4fc4-b520-0ec2cb0c10e0" containerName="main"
Jul 07 00:32:01.227338 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.227112    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a77db02-a1c4-4f38-9dd2-8104e2a5b2b8" containerName="cleanup"
Jul 07 00:32:01.236596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.236561    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.239878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.239843    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-259681-openshift-default-dockercfg-cz75j\""
Jul 07 00:32:01.243216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.243179    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:01.250129 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.250103    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:01.351183 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351142    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351183 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351188    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351216    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmzkm\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351287    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351395    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351432    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351611 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351494    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.351611 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.351546    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.352942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.352923    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:01.452555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452516    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452555    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452762 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452577    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452869 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452920 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452984 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452955    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.452984 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.452970    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453078 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453026    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453164 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453141    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453332 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmzkm\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453423 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453340    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453423 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453360    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.453423 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.453314    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.455032 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.455011    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.455219 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.455202    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.461152 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.461126    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmzkm\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.461231 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.461186    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token\") pod \"e2e-api-a-259681-openshift-default-5774c4657-lx6fq\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") " pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.550497 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.550408    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:01.695884 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.695856    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:01.698214 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:32:01.698189    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63e80b6_77ef_46ff_8b6c_424d7d24672e.slice/crio-2f69a906e0bdc753d70234fcbcc269253f45851327c0e0cc6b0cd951bcab5945 WatchSource:0}: Error finding container 2f69a906e0bdc753d70234fcbcc269253f45851327c0e0cc6b0cd951bcab5945: Status 404 returned error can't find the container with id 2f69a906e0bdc753d70234fcbcc269253f45851327c0e0cc6b0cd951bcab5945
Jul 07 00:32:01.700328 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.700277    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:01.700419 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.700361    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:01.700419 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.700389    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:01.858774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.858690    2583 patch_prober.go:28] interesting pod/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 07 00:32:01.858774 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:01.858749    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 07 00:32:02.176940 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:02.176904    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerStarted","Data":"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"}
Jul 07 00:32:02.177451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:02.176961    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerStarted","Data":"2f69a906e0bdc753d70234fcbcc269253f45851327c0e0cc6b0cd951bcab5945"}
Jul 07 00:32:02.196643 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:02.196594    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podStartSLOduration=1.196573193 podStartE2EDuration="1.196573193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:02.195553041 +0000 UTC m=+2107.520556324" watchObservedRunningTime="2026-07-07 00:32:02.196573193 +0000 UTC m=+2107.521576601"
Jul 07 00:32:02.551256 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:02.551161    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:02.556117 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:02.556088    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:03.050384 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:03.050350    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:03.150850 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:03.150813    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:03.181354 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:03.181318    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:03.182389 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:03.182373    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:04.188794 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:04.188762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:04.266158 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:04.266127    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:04.334095 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:04.334058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:04.473532 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:04.473434    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:04.899371 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:04.899338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:05.033112 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.033075    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:05.544791 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.544721    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:32:05.696876 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.696841    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.696876 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.696882    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.696921    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.696963    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697003    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697027    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28mm2\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697075    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697102 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697095    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697129    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert\") pod \"111ddc50-80ee-4631-a699-540aff53f12e\" (UID: \"111ddc50-80ee-4631-a699-540aff53f12e\") "
Jul 07 00:32:05.697427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697236    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:05.697427 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697400    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:05.697592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697486    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.697592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697505    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.697739 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697717    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:05.697824 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697802    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:32:05.697878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.697842    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data" (OuterVolumeSpecName: "istio-data") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:05.699212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.699178    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:05.699379 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.699260    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2" (OuterVolumeSpecName: "kube-api-access-28mm2") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "kube-api-access-28mm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:05.699573 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.699556    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:32:05.699636 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.699624    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token" (OuterVolumeSpecName: "istio-token") pod "111ddc50-80ee-4631-a699-540aff53f12e" (UID: "111ddc50-80ee-4631-a699-540aff53f12e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798717    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/111ddc50-80ee-4631-a699-540aff53f12e-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798745    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798756    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/111ddc50-80ee-4631-a699-540aff53f12e-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798766    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798776    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798786    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/111ddc50-80ee-4631-a699-540aff53f12e-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:05.798810 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:05.798799    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28mm2\" (UniqueName: \"kubernetes.io/projected/111ddc50-80ee-4631-a699-540aff53f12e-kube-api-access-28mm2\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:06.197696 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.197656    2583 generic.go:356] "Generic (PLEG): container finished" podID="111ddc50-80ee-4631-a699-540aff53f12e" containerID="a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226" exitCode=0
Jul 07 00:32:06.197899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.197728    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"
Jul 07 00:32:06.197899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.197741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerDied","Data":"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"}
Jul 07 00:32:06.197899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.197803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n" event={"ID":"111ddc50-80ee-4631-a699-540aff53f12e","Type":"ContainerDied","Data":"4da6b9279b2a27e88ca685b5531f0c57897f44b61229b319fcd5147a910e70f6"}
Jul 07 00:32:06.197899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.197828    2583 scope.go:117] "RemoveContainer" containerID="a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"
Jul 07 00:32:06.207357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.207340    2583 scope.go:117] "RemoveContainer" containerID="af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"
Jul 07 00:32:06.215088 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.215068    2583 scope.go:117] "RemoveContainer" containerID="a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"
Jul 07 00:32:06.215353 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:32:06.215331    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226\": container with ID starting with a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226 not found: ID does not exist" containerID="a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"
Jul 07 00:32:06.215425 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.215365    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226"} err="failed to get container status \"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226\": rpc error: code = NotFound desc = could not find container \"a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226\": container with ID starting with a7642fa8219d61416233b4b893cfab5c7d7457849aa2b0e77a937ac687982226 not found: ID does not exist"
Jul 07 00:32:06.215425 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.215382    2583 scope.go:117] "RemoveContainer" containerID="af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"
Jul 07 00:32:06.215644 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:32:06.215620    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7\": container with ID starting with af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7 not found: ID does not exist" containerID="af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"
Jul 07 00:32:06.215705 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.215653    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7"} err="failed to get container status \"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7\": rpc error: code = NotFound desc = could not find container \"af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7\": container with ID starting with af6a68d4654e2b76a0ae214320c01f24a66edac07e5b192f8348f1b8b343b5d7 not found: ID does not exist"
Jul 07 00:32:06.220761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.220738    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:32:06.229200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:06.229179    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-dd30cd-openshift-default-74bfcc8f9f-hqq2n"]
Jul 07 00:32:07.158955 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:07.158923    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="111ddc50-80ee-4631-a699-540aff53f12e" path="/var/lib/kubelet/pods/111ddc50-80ee-4631-a699-540aff53f12e/volumes"
Jul 07 00:32:08.715981 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:08.715950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:09.195664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.195634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:09.492020 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.491941    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:09.492337 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492323    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.492380 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492338    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.492380 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492353    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.492380 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492359    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.492475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492442    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="845e1939-9ee9-42d7-b97e-fdfbf1fd62cc" containerName="istio-proxy"
Jul 07 00:32:09.492475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492461    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.492475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.492469    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="111ddc50-80ee-4631-a699-540aff53f12e" containerName="istio-proxy"
Jul 07 00:32:09.495920 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.495901    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.498583 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.498566    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-26d2ae-openshift-default-dockercfg-lvx77\""
Jul 07 00:32:09.507803 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.507779    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:09.531764 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.531736    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:09.639591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639557    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639642    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639768 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639737    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639823    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639842    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk7gf\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.639910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.639864    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741199 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741255    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk7gf\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741347    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741467    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.741717 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741591    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.742191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741798    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.742191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.741985    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.742191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.742026    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.742191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.742054    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.742191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.742093    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.744087 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.744065    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.744191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.744102    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.754666 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.754642    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.756475 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.756446    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk7gf\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf\") pod \"e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") " pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.772140 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.772106    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:09.806217 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.806192    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:09.952950 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.952926    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:09.954568 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:32:09.954541    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243f6ba2_54d3_4816_bfa0_1fca67cbc01a.slice/crio-312bdd0da2800cdfa2517c0bbbacda6c3846bee8b1c5ae1e2fb9a088ce8ec418 WatchSource:0}: Error finding container 312bdd0da2800cdfa2517c0bbbacda6c3846bee8b1c5ae1e2fb9a088ce8ec418: Status 404 returned error can't find the container with id 312bdd0da2800cdfa2517c0bbbacda6c3846bee8b1c5ae1e2fb9a088ce8ec418
Jul 07 00:32:09.956695 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.956662    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:09.956757 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.956732    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:09.956799 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:09.956763    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:10.218088 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:10.218051    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerStarted","Data":"bb485214df06bb18327637cabf304ef53c07aa01ab413b497e578a80b12e195b"}
Jul 07 00:32:10.218250 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:10.218108    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerStarted","Data":"312bdd0da2800cdfa2517c0bbbacda6c3846bee8b1c5ae1e2fb9a088ce8ec418"}
Jul 07 00:32:10.238092 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:10.238040    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podStartSLOduration=1.238025387 podStartE2EDuration="1.238025387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:10.236001859 +0000 UTC m=+2115.561005144" watchObservedRunningTime="2026-07-07 00:32:10.238025387 +0000 UTC m=+2115.563028672"
Jul 07 00:32:10.807259 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:10.807225    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:11.812439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:11.812407    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:12.182866 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.182833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:12.230169 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.230128    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:12.230997 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.230979    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:12.261817 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.261782    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:12.470900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.470815    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:12.775306 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:12.775203    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:13.124647 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:13.124611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:13.225953 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:13.225925    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:16.710072 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:16.710042    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:19.223474 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:19.223431    2583 patch_prober.go:28] interesting pod/e2e-api-a-259681-openshift-default-5774c4657-lx6fq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 00:32:19.223879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:19.223542    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:32:20.286457 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.286424    2583 generic.go:356] "Generic (PLEG): container finished" podID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerID="48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab" exitCode=0
Jul 07 00:32:20.286876 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.286495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerDied","Data":"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"}
Jul 07 00:32:20.286920 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.286910    2583 scope.go:117] "RemoveContainer" containerID="48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"
Jul 07 00:32:20.287911 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.287884    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:20.287965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.287953    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:20.288011 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:20.287983    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:21.293389 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:21.293347    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerStarted","Data":"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"}
Jul 07 00:32:21.551603 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:21.551081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:21.551603 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:21.551218    2583 patch_prober.go:28] interesting pod/e2e-api-a-259681-openshift-default-5774c4657-lx6fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 07 00:32:21.551603 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:21.551262    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 07 00:32:22.551655 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:22.551615    2583 patch_prober.go:28] interesting pod/e2e-api-a-259681-openshift-default-5774c4657-lx6fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 07 00:32:22.552722 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:22.552253    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 07 00:32:23.550915 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:23.550867    2583 patch_prober.go:28] interesting pod/e2e-api-a-259681-openshift-default-5774c4657-lx6fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 07 00:32:23.551145 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:23.550957    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 07 00:32:24.551916 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:24.551400    2583 patch_prober.go:28] interesting pod/e2e-api-a-259681-openshift-default-5774c4657-lx6fq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 07 00:32:24.551916 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:24.551468    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 07 00:32:25.084777 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.084740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:25.407354 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.407314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:25.509566 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.509531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:25.554597 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.554563    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:25.555042 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.554956    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:25.555611 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.555593    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:25.605966 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.605932    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:25.823181 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:25.823090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:28.262555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.262120    2583 patch_prober.go:28] interesting pod/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 07 00:32:28.262555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.262195    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 07 00:32:28.333919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.333873    2583 generic.go:356] "Generic (PLEG): container finished" podID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerID="bb485214df06bb18327637cabf304ef53c07aa01ab413b497e578a80b12e195b" exitCode=0
Jul 07 00:32:28.334326 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.333947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerDied","Data":"bb485214df06bb18327637cabf304ef53c07aa01ab413b497e578a80b12e195b"}
Jul 07 00:32:28.334464 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.334446    2583 scope.go:117] "RemoveContainer" containerID="bb485214df06bb18327637cabf304ef53c07aa01ab413b497e578a80b12e195b"
Jul 07 00:32:28.335536 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.335510    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:28.335596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.335579    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:28.335631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:28.335610    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:29.345488 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:29.345449    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerStarted","Data":"07a7cb5caf7cfc4debf692a51823d68730a375c34c7f7669a5e3e1439c7bfe4b"}
Jul 07 00:32:29.806348 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:29.806309    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:29.806642 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:29.806619    2583 patch_prober.go:28] interesting pod/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 07 00:32:29.806739 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:29.806689    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 07 00:32:30.806548 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:30.806514    2583 patch_prober.go:28] interesting pod/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 07 00:32:30.806942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:30.806569    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 07 00:32:31.200894 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.200864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:31.262420 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.262368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:31.454404 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.454269    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:31.454927 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.454880    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy" containerID="cri-o://5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452" gracePeriod=30
Jul 07 00:32:31.476651 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.476624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:31.806991 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.806902    2583 patch_prober.go:28] interesting pod/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 07 00:32:31.806991 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.806957    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 07 00:32:31.863478 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:31.863444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:32.298773 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:32.298746    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:32.399085 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:32.399036    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:32.571766 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:32.571681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:32.807204 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:32.807166    2583 patch_prober.go:28] interesting pod/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 07 00:32:32.807662 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:32.807236    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 07 00:32:33.810284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:33.810250    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:33.810760 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:33.810741    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:33.811349 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:33.811331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:36.485668 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:32:36.485641    2583 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63e80b6_77ef_46ff_8b6c_424d7d24672e.slice/crio-conmon-5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452.scope\": RecentStats: unable to find data in memory cache]"
Jul 07 00:32:36.720340 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.720315    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:36.823891 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.823810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.823891 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.823859    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.823894    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.823924    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.823972    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824010    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmzkm\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824062    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824090    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824117    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data\") pod \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\" (UID: \"c63e80b6-77ef-46ff-8b6c-424d7d24672e\") "
Jul 07 00:32:36.824392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824246    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:36.824555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824502    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.824555 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824535    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:36.824670 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824562    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:32:36.824729 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.824700    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:36.825159 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.825127    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data" (OuterVolumeSpecName: "istio-data") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:36.826191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.826165    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:36.826314 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.826216    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token" (OuterVolumeSpecName: "istio-token") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:36.826314 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.826235    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:32:36.826404 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.826372    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm" (OuterVolumeSpecName: "kube-api-access-pmzkm") pod "c63e80b6-77ef-46ff-8b6c-424d7d24672e" (UID: "c63e80b6-77ef-46ff-8b6c-424d7d24672e"). InnerVolumeSpecName "kube-api-access-pmzkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:36.925370 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925335    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925370 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925365    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925370 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925374    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925383    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925391    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmzkm\" (UniqueName: \"kubernetes.io/projected/c63e80b6-77ef-46ff-8b6c-424d7d24672e-kube-api-access-pmzkm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925400    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925410    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:36.925607 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:36.925417    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c63e80b6-77ef-46ff-8b6c-424d7d24672e-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:37.398447 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.398412    2583 generic.go:356] "Generic (PLEG): container finished" podID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerID="5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452" exitCode=0
Jul 07 00:32:37.398629 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.398463    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"
Jul 07 00:32:37.398629 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.398493    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerDied","Data":"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"}
Jul 07 00:32:37.398629 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.398552    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq" event={"ID":"c63e80b6-77ef-46ff-8b6c-424d7d24672e","Type":"ContainerDied","Data":"2f69a906e0bdc753d70234fcbcc269253f45851327c0e0cc6b0cd951bcab5945"}
Jul 07 00:32:37.398629 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.398583    2583 scope.go:117] "RemoveContainer" containerID="5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"
Jul 07 00:32:37.408901 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.408285    2583 scope.go:117] "RemoveContainer" containerID="48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"
Jul 07 00:32:37.418062 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.418039    2583 scope.go:117] "RemoveContainer" containerID="5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"
Jul 07 00:32:37.418210 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.418188    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:37.418398 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:32:37.418377    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452\": container with ID starting with 5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452 not found: ID does not exist" containerID="5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"
Jul 07 00:32:37.418451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.418407    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452"} err="failed to get container status \"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452\": rpc error: code = NotFound desc = could not find container \"5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452\": container with ID starting with 5eae34bba9865de6a4f5afe89ff13b1ac0b27c40d47976e95c4bb254e769a452 not found: ID does not exist"
Jul 07 00:32:37.418451 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.418428    2583 scope.go:117] "RemoveContainer" containerID="48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"
Jul 07 00:32:37.418676 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:32:37.418658    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab\": container with ID starting with 48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab not found: ID does not exist" containerID="48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"
Jul 07 00:32:37.418720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.418683    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab"} err="failed to get container status \"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab\": rpc error: code = NotFound desc = could not find container \"48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab\": container with ID starting with 48c0b170b5fae0fbe1660bea4e8a8c7f136457c8ab6c3f78e5bc1744cc78c9ab not found: ID does not exist"
Jul 07 00:32:37.422056 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:37.422032    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-259681-openshift-default-5774c4657-lx6fq"]
Jul 07 00:32:38.095163 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.095120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:38.199074 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.199038    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:38.324522 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.324493    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:38.324737 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.324712    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy" containerID="cri-o://07a7cb5caf7cfc4debf692a51823d68730a375c34c7f7669a5e3e1439c7bfe4b" gracePeriod=30
Jul 07 00:32:38.363340 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.363208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:38.908676 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:38.908646    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:39.159568 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.159474    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" path="/var/lib/kubelet/pods/c63e80b6-77ef-46ff-8b6c-424d7d24672e/volumes"
Jul 07 00:32:39.219337 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219303    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:32:39.219686 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219674    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.219745 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219688    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.219745 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219704    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.219745 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219710    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.219836 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219769    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.219836 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.219777    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c63e80b6-77ef-46ff-8b6c-424d7d24672e" containerName="istio-proxy"
Jul 07 00:32:39.223782 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.223765    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.226407 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.226386    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-a8caec-openshift-default-dockercfg-cjq2k\""
Jul 07 00:32:39.236878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.236854    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:32:39.253467 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.253432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:39.349381 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349345    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdlfj\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349381 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349384    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349411    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349429    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349511    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349547    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349585 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349579    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.349776 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.349611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.438287 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.438199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:39.450273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450237    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdlfj\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450273 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450272    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450326    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450456    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450480    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450524 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450524    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450784 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450563    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450784 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450593    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450784 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450720    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450789    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450839    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.450928 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.450874    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.451161 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.451139    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.452841 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.452815    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.453072 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.453052    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.459010 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.458988    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.459173 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.459155    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdlfj\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj\") pod \"e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") " pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.537230 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.537190    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:39.679482 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.679457    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:32:39.682052 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:32:39.682023    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2877e0ec_e4ec_4299_9e90_073ab11e00c4.slice/crio-78ee947ad385fe902e854f8143c595a3f8d354e8f0114d05760dea784252ea8d WatchSource:0}: Error finding container 78ee947ad385fe902e854f8143c595a3f8d354e8f0114d05760dea784252ea8d: Status 404 returned error can't find the container with id 78ee947ad385fe902e854f8143c595a3f8d354e8f0114d05760dea784252ea8d
Jul 07 00:32:39.684249 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.684217    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:39.684357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.684287    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:39.684406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:39.684385    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:40.423033 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:40.422993    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerStarted","Data":"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"}
Jul 07 00:32:40.423465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:40.423057    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerStarted","Data":"78ee947ad385fe902e854f8143c595a3f8d354e8f0114d05760dea784252ea8d"}
Jul 07 00:32:40.444160 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:40.444112    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podStartSLOduration=1.4440934460000001 podStartE2EDuration="1.444093446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:40.443390601 +0000 UTC m=+2145.768393887" watchObservedRunningTime="2026-07-07 00:32:40.444093446 +0000 UTC m=+2145.769096787"
Jul 07 00:32:40.537612 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:40.537570    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:40.542583 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:40.542558    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:41.428530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:41.428492    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:41.429710 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:41.429684    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:42.405218 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:42.405183    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:42.526561 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:42.526530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:42.634083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:42.634003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:43.034617 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.034580    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:43.404824 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.404790    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:43.443001 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.442975    2583 generic.go:356] "Generic (PLEG): container finished" podID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerID="07a7cb5caf7cfc4debf692a51823d68730a375c34c7f7669a5e3e1439c7bfe4b" exitCode=0
Jul 07 00:32:43.443150 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.443044    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerDied","Data":"07a7cb5caf7cfc4debf692a51823d68730a375c34c7f7669a5e3e1439c7bfe4b"}
Jul 07 00:32:43.443150 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.443103    2583 scope.go:117] "RemoveContainer" containerID="bb485214df06bb18327637cabf304ef53c07aa01ab413b497e578a80b12e195b"
Jul 07 00:32:43.505006 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.504975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:43.584491 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.584433    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:43.692679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692633    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.692878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692733    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.692878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692769    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.692878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692890    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692940    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.692973    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fk7gf\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693047 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693022    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693268 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693115    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs\") pod \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\" (UID: \"243f6ba2-54d3-4816-bfa0-1fca67cbc01a\") "
Jul 07 00:32:43.693596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693195    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:43.693596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693274    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:32:43.693596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693425    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:43.693596 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693560    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:43.693818 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693800    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.693818 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693804    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data" (OuterVolumeSpecName: "istio-data") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:43.693889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693824    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.693889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693841    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.693889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.693856    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.695279 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.695249    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:32:43.695428 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.695335    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:32:43.695485 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.695441    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf" (OuterVolumeSpecName: "kube-api-access-fk7gf") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "kube-api-access-fk7gf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:43.695564 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.695541    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token" (OuterVolumeSpecName: "istio-token") pod "243f6ba2-54d3-4816-bfa0-1fca67cbc01a" (UID: "243f6ba2-54d3-4816-bfa0-1fca67cbc01a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:32:43.795238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.795194    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.795238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.795230    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.795238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.795241    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.795512 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.795254    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk7gf\" (UniqueName: \"kubernetes.io/projected/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-kube-api-access-fk7gf\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:43.795512 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:43.795268    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/243f6ba2-54d3-4816-bfa0-1fca67cbc01a-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:32:44.448965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:44.448933    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c" event={"ID":"243f6ba2-54d3-4816-bfa0-1fca67cbc01a","Type":"ContainerDied","Data":"312bdd0da2800cdfa2517c0bbbacda6c3846bee8b1c5ae1e2fb9a088ce8ec418"}
Jul 07 00:32:44.448965 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:44.448952    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"
Jul 07 00:32:44.449244 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:44.449000    2583 scope.go:117] "RemoveContainer" containerID="07a7cb5caf7cfc4debf692a51823d68730a375c34c7f7669a5e3e1439c7bfe4b"
Jul 07 00:32:44.474241 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:44.474203    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:44.479141 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:44.479108    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-26d2ae-openshift-default-657cd886fc-pk29c"]
Jul 07 00:32:45.159722 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:45.159689    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" path="/var/lib/kubelet/pods/243f6ba2-54d3-4816-bfa0-1fca67cbc01a/volumes"
Jul 07 00:32:46.907408 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:46.907319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:47.243694 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.243592    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:47.512906 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.512816    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:32:47.513231 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513217    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.513357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513232    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.513357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513251    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.513357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513256    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.513357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513339    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.513357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.513349    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="243f6ba2-54d3-4816-bfa0-1fca67cbc01a" containerName="istio-proxy"
Jul 07 00:32:47.517985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.517953    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.520948 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.520925    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-3ee067-openshift-default-dockercfg-dl5rj\""
Jul 07 00:32:47.528269 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.528244    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:32:47.545530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.545469    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:47.635505 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635468    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635550    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635582    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635662    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.635720 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7ckh\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.636074 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.635727    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.646067 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.646021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:47.736703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.736669    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.736703 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.736710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.736979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.736732    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.736979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.736762    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7ckh\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.736979 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.736819    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737127 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737005    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737127 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737084    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737127 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737116    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737264 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737125    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737264 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737176    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737264 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737175    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737415    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737571    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.737728 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.737711    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.739449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.739430    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.739560 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.739489    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.748847 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.748818    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7ckh\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.749252 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.749229    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token\") pod \"e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") " pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.830061 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.829967    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:47.987646 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.987620    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:32:47.988629 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:32:47.988604    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd93be0_37e8_4c46_960f_a7c34bd8e837.slice/crio-7b212a78c5cdb31a329dcbecfd50611613bfbca8a3ed8b7170e0cba1089e5a1d WatchSource:0}: Error finding container 7b212a78c5cdb31a329dcbecfd50611613bfbca8a3ed8b7170e0cba1089e5a1d: Status 404 returned error can't find the container with id 7b212a78c5cdb31a329dcbecfd50611613bfbca8a3ed8b7170e0cba1089e5a1d
Jul 07 00:32:47.990784 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.990747    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:47.990858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.990816    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:47.990858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:47.990853    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:48.471940 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:48.471906    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerStarted","Data":"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"}
Jul 07 00:32:48.472122 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:48.471963    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerStarted","Data":"7b212a78c5cdb31a329dcbecfd50611613bfbca8a3ed8b7170e0cba1089e5a1d"}
Jul 07 00:32:48.501546 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:48.501493    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podStartSLOduration=1.5014736210000001 podStartE2EDuration="1.501473621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:32:48.501116282 +0000 UTC m=+2153.826119569" watchObservedRunningTime="2026-07-07 00:32:48.501473621 +0000 UTC m=+2153.826476905"
Jul 07 00:32:48.830752 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:48.830653    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:49.862375 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:49.862334    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:50.482754 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.482721    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:50.483761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.483740    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:32:50.527135 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.527092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:50.600881 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.600849    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:50.699633 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.699600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:50.858192 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:50.858097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:51.168212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:51.168179    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:51.267754 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:51.267714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:55.009386 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:55.009348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:32:56.430859 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.430807    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:32:56.431467 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.430903    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:32:56.512553 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.512522    2583 generic.go:356] "Generic (PLEG): container finished" podID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerID="8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff" exitCode=0
Jul 07 00:32:56.512718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.512570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerDied","Data":"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"}
Jul 07 00:32:56.512966 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.512952    2583 scope.go:117] "RemoveContainer" containerID="8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"
Jul 07 00:32:56.513999 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.513967    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:56.514062 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.514036    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:56.514104 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:56.514073    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:32:57.519759 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:57.519725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerStarted","Data":"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"}
Jul 07 00:32:57.537763 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:57.537696    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:32:57.538106 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:57.538075    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:32:57.538212 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:57.538142    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:32:58.538156 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:58.538112    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:32:58.538684 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:58.538177    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:32:59.537985 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:59.537940    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:32:59.538201 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:32:59.538011    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:33:00.538369 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:00.537828    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:33:00.538369 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:00.537925    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:33:01.538761 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:01.538720    2583 patch_prober.go:28] interesting pod/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 07 00:33:01.539357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:01.538786    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 07 00:33:02.541485 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:02.541451    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:33:02.541898 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:02.541809    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:33:02.542653 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:02.542634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:33:03.554825 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.554739    2583 generic.go:356] "Generic (PLEG): container finished" podID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerID="471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd" exitCode=0
Jul 07 00:33:03.555247 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.554814    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerDied","Data":"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"}
Jul 07 00:33:03.555338 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.555324    2583 scope.go:117] "RemoveContainer" containerID="471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"
Jul 07 00:33:03.556260 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.556231    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:33:03.556392 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.556333    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:33:03.556463 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:03.556407    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 07 00:33:04.563311 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.563252    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerStarted","Data":"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"}
Jul 07 00:33:04.736070 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.736000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:04.831788 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.831689    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:33:04.831788 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.831739    2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 07 00:33:04.832043 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.831802    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 07 00:33:04.836878 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:04.836841    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:05.482766 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:05.482731    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:33:05.831251 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:05.831147    2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 07 00:33:05.832434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:05.832398    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 07 00:33:06.830795 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:06.830744    2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 07 00:33:06.831023 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:06.830832    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 07 00:33:07.145912 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:07.145863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:07.830580 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:07.830532    2583 patch_prober.go:28] interesting pod/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 07 00:33:07.830788 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:07.830648    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 07 00:33:08.834809 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:08.834766    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:33:08.835838 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:08.835818    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:33:12.840637 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:12.840601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:12.942122 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:12.942083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:26.301805 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.301770    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:33:26.305155 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.305133    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.309006 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.308967    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"openshift-service-ca.crt\""
Jul 07 00:33:26.309006 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.308986    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"auth-test-model-3ee067-kserve-self-signed-certs\""
Jul 07 00:33:26.309200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.308979    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"default-dockercfg-wtwq6\""
Jul 07 00:33:26.309200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.309044    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"kube-root-ca.crt\""
Jul 07 00:33:26.321003 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.320977    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:33:26.350421 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.350382    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:26.439196 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439159    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx7n5\" (UniqueName: \"kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.439398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439221    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.439398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439347    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.439398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439391    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.439527 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.439527 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.439460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540413 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540372    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx7n5\" (UniqueName: \"kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540638 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540638 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540484    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540638 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540510    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540638 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540526    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540834 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540651    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.540970 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540950    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.541034 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.540998    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.541083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.541045    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.542866 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.542845    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.542975 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.542957    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.549019 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.548993    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx7n5\" (UniqueName: \"kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5\") pod \"auth-test-model-3ee067-kserve-78df45fb67-p6szb\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") " pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.615436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.615331    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:26.757503 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:26.757469    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:33:26.759405 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:33:26.759363    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod825bb0dc_cd1f_4d00_8a76_4bbfb9f2d608.slice/crio-95b6e239c4bb899283fd07257db41ad734e67b42b9114c4b3ec1c0708a9eb97a WatchSource:0}: Error finding container 95b6e239c4bb899283fd07257db41ad734e67b42b9114c4b3ec1c0708a9eb97a: Status 404 returned error can't find the container with id 95b6e239c4bb899283fd07257db41ad734e67b42b9114c4b3ec1c0708a9eb97a
Jul 07 00:33:27.683264 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:27.683229    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" event={"ID":"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608","Type":"ContainerStarted","Data":"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"}
Jul 07 00:33:27.683264 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:27.683268    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" event={"ID":"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608","Type":"ContainerStarted","Data":"95b6e239c4bb899283fd07257db41ad734e67b42b9114c4b3ec1c0708a9eb97a"}
Jul 07 00:33:27.703342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:27.703274    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" podStartSLOduration=1.703260347 podStartE2EDuration="1.703260347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:33:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:33:27.70153782 +0000 UTC m=+2193.026541108" watchObservedRunningTime="2026-07-07 00:33:27.703260347 +0000 UTC m=+2193.028263631"
Jul 07 00:33:36.616286 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:36.616238    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:36.616827 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:36.616332    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:36.628466 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:36.628436    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:36.739697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:36.739665    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:33:38.699477 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:38.699439    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:57.656807 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:57.656767    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:58.977180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:58.977148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:33:59.035004 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:33:59.034965    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:17.005815 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:17.005778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:17.898953 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:17.898920    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:35:17.918946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:17.918899    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:22.438191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:22.438158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:32.919910 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:32.919863    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" podUID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" containerName="main" containerID="cri-o://5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38" gracePeriod=45
Jul 07 00:35:33.178409 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.178350    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:35:33.314681 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.314648    2583 generic.go:356] "Generic (PLEG): container finished" podID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" containerID="5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38" exitCode=0
Jul 07 00:35:33.314858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.314689    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"
Jul 07 00:35:33.314858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.314715    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" event={"ID":"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608","Type":"ContainerDied","Data":"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"}
Jul 07 00:35:33.314858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.314749    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb" event={"ID":"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608","Type":"ContainerDied","Data":"95b6e239c4bb899283fd07257db41ad734e67b42b9114c4b3ec1c0708a9eb97a"}
Jul 07 00:35:33.314858 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.314767    2583 scope.go:117] "RemoveContainer" containerID="5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"
Jul 07 00:35:33.322963 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.322944    2583 scope.go:117] "RemoveContainer" containerID="5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"
Jul 07 00:35:33.323240 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:35:33.323210    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38\": container with ID starting with 5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38 not found: ID does not exist" containerID="5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"
Jul 07 00:35:33.323357 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.323245    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38"} err="failed to get container status \"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38\": rpc error: code = NotFound desc = could not find container \"5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38\": container with ID starting with 5051217ad5ec099f4f5a7b4f047ca16d7a186d4fba2cc36e84e0dd15a1537d38 not found: ID does not exist"
Jul 07 00:35:33.366696 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366661    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx7n5\" (UniqueName: \"kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.366870 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366703    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.366870 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366732    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.366870 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366821    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.366870 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366852    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.367079 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.366876    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache\") pod \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\" (UID: \"825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608\") "
Jul 07 00:35:33.367079 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.367032    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home" (OuterVolumeSpecName: "home") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:33.367201 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.367135    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:33.367265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.367192    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache" (OuterVolumeSpecName: "model-cache") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:33.367265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.367235    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-home\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.367265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.367254    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tmp-dir\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.368956 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.368927    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:35:33.368956 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.368937    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm" (OuterVolumeSpecName: "dshm") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:35:33.369119 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.369029    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5" (OuterVolumeSpecName: "kube-api-access-dx7n5") pod "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" (UID: "825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608"). InnerVolumeSpecName "kube-api-access-dx7n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:35:33.468718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.468613    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx7n5\" (UniqueName: \"kubernetes.io/projected/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-kube-api-access-dx7n5\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.468718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.468658    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-dshm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.468718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.468669    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-tls-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.468718 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.468678    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608-model-cache\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:35:33.638443 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.638412    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:35:33.643649 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:33.643624    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/auth-test-model-3ee067-kserve-78df45fb67-p6szb"]
Jul 07 00:35:35.159465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:35.159435    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" path="/var/lib/kubelet/pods/825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608/volumes"
Jul 07 00:35:39.555970 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.555938    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:35:39.556372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.556359    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" containerName="main"
Jul 07 00:35:39.556372 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.556373    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" containerName="main"
Jul 07 00:35:39.556455 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.556449    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="825bb0dc-cd1f-4d00-8a76-4bbfb9f2d608" containerName="main"
Jul 07 00:35:39.560722 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.560697    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.563481 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.563460    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"default-dockercfg-wtwq6\""
Jul 07 00:35:39.564591 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.564570    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"sub-test-model-3ee067-kserve-self-signed-certs\""
Jul 07 00:35:39.564675 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.564603    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"openshift-service-ca.crt\""
Jul 07 00:35:39.564675 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.564570    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"kube-root-ca.crt\""
Jul 07 00:35:39.573707 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.573680    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:35:39.592180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.592147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:35:39.627725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.627694    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86zxf\" (UniqueName: \"kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.627895 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.627745    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.627962 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.627940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.628018 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.627988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.628063 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.628030    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.628108 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.628084    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.728749 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.728708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.728940 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.728776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.728940 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.728825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86zxf\" (UniqueName: \"kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.728940 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.728864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.729148 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.728966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.729148 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.729001    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.729342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.729188    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.729342 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.729333    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.729543 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.729502    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.731473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.731445    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.731473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.731468    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.753187 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.753155    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86zxf\" (UniqueName: \"kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf\") pod \"sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") " pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:39.870982 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:39.870952    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:40.012678 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:40.012650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:35:40.014269 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:35:40.014238    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d83c605_5f0e_4097_932b_18c422ae1351.slice/crio-f950fa58907990e03f20e5180f6c87f18138cd0b5b9ecdfcdeb5a21491c08b2c WatchSource:0}: Error finding container f950fa58907990e03f20e5180f6c87f18138cd0b5b9ecdfcdeb5a21491c08b2c: Status 404 returned error can't find the container with id f950fa58907990e03f20e5180f6c87f18138cd0b5b9ecdfcdeb5a21491c08b2c
Jul 07 00:35:40.352852 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:40.352752    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" event={"ID":"9d83c605-5f0e-4097-932b-18c422ae1351","Type":"ContainerStarted","Data":"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"}
Jul 07 00:35:40.352852 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:40.352795    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" event={"ID":"9d83c605-5f0e-4097-932b-18c422ae1351","Type":"ContainerStarted","Data":"f950fa58907990e03f20e5180f6c87f18138cd0b5b9ecdfcdeb5a21491c08b2c"}
Jul 07 00:35:40.369265 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:40.369212    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" podStartSLOduration=1.369198131 podStartE2EDuration="1.369198131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:35:40.368617045 +0000 UTC m=+2325.693620330" watchObservedRunningTime="2026-07-07 00:35:40.369198131 +0000 UTC m=+2325.694201415"
Jul 07 00:35:49.871900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:49.871797    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:49.871900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:49.871853    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:49.884674 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:49.884640    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:35:50.413837 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:35:50.413806    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:36:14.648284 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:14.648247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:15.700062 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:15.700026    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:36:15.722472 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:15.722433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:24.622128 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:24.622096    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:36:30.726184 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:30.726141    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" podUID="9d83c605-5f0e-4097-932b-18c422ae1351" containerName="main" containerID="cri-o://4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549" gracePeriod=45
Jul 07 00:36:30.984439 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:30.984372    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:36:31.030397 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030368    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030414    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030454    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86zxf\" (UniqueName: \"kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030514    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030545    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030593 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030589    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home\") pod \"9d83c605-5f0e-4097-932b-18c422ae1351\" (UID: \"9d83c605-5f0e-4097-932b-18c422ae1351\") "
Jul 07 00:36:31.030986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030832    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:31.030986 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030940    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-model-cache\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.031163 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.030939    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:31.031163 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.031029    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home" (OuterVolumeSpecName: "home") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:31.032692 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.032669    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:36:31.032971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.032952    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm" (OuterVolumeSpecName: "dshm") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:36:31.033145 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.033125    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf" (OuterVolumeSpecName: "kube-api-access-86zxf") pod "9d83c605-5f0e-4097-932b-18c422ae1351" (UID: "9d83c605-5f0e-4097-932b-18c422ae1351"). InnerVolumeSpecName "kube-api-access-86zxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:36:31.132530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.132487    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-dshm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.132530 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.132526    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86zxf\" (UniqueName: \"kubernetes.io/projected/9d83c605-5f0e-4097-932b-18c422ae1351-kube-api-access-86zxf\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.132746 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.132540    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-tmp-dir\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.132746 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.132551    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d83c605-5f0e-4097-932b-18c422ae1351-home\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.132746 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.132561    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d83c605-5f0e-4097-932b-18c422ae1351-tls-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:36:31.613363 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.613323    2583 generic.go:356] "Generic (PLEG): container finished" podID="9d83c605-5f0e-4097-932b-18c422ae1351" containerID="4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549" exitCode=0
Jul 07 00:36:31.613535 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.613385    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"
Jul 07 00:36:31.613535 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.613397    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" event={"ID":"9d83c605-5f0e-4097-932b-18c422ae1351","Type":"ContainerDied","Data":"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"}
Jul 07 00:36:31.613535 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.613438    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr" event={"ID":"9d83c605-5f0e-4097-932b-18c422ae1351","Type":"ContainerDied","Data":"f950fa58907990e03f20e5180f6c87f18138cd0b5b9ecdfcdeb5a21491c08b2c"}
Jul 07 00:36:31.613535 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.613454    2583 scope.go:117] "RemoveContainer" containerID="4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"
Jul 07 00:36:31.622754 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.622729    2583 scope.go:117] "RemoveContainer" containerID="4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"
Jul 07 00:36:31.623033 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:36:31.623015    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549\": container with ID starting with 4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549 not found: ID does not exist" containerID="4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"
Jul 07 00:36:31.623088 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.623043    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549"} err="failed to get container status \"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549\": rpc error: code = NotFound desc = could not find container \"4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549\": container with ID starting with 4aed565dc4c7c08b4fad5fc26830dce6eb44cf0adeed7fbba3dc1cbc51dd2549 not found: ID does not exist"
Jul 07 00:36:31.631500 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.631474    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:36:31.635609 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:31.635587    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/sub-test-model-3ee067-kserve-5785fb5c6c-z5xnr"]
Jul 07 00:36:33.158643 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:33.158610    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d83c605-5f0e-4097-932b-18c422ae1351" path="/var/lib/kubelet/pods/9d83c605-5f0e-4097-932b-18c422ae1351/volumes"
Jul 07 00:36:55.335228 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:55.335190    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:36:55.345135 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:36:55.345113    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:37:03.861756 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:03.861720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:03.964031 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:03.963998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:15.097694 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.097659    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:37:15.098266 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.098252    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d83c605-5f0e-4097-932b-18c422ae1351" containerName="main"
Jul 07 00:37:15.098375 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.098273    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d83c605-5f0e-4097-932b-18c422ae1351" containerName="main"
Jul 07 00:37:15.098434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.098420    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d83c605-5f0e-4097-932b-18c422ae1351" containerName="main"
Jul 07 00:37:15.101700 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.101678    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.105462 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.105433    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"openshift-service-ca.crt\""
Jul 07 00:37:15.105574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.105484    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"default-dockercfg-wtwq6\""
Jul 07 00:37:15.105574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.105498    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"kube-root-ca.crt\""
Jul 07 00:37:15.105574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.105431    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3ee067\"/\"rate-test-model-3ee067-kserve-self-signed-certs\""
Jul 07 00:37:15.109961 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.109942    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:37:15.145566 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.145686 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.145686 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145646    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9xv\" (UniqueName: \"kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.145806 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.145862 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145819    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.145862 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.145853    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.188441 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.188407    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:15.246899 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.246869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9xv\" (UniqueName: \"kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.246910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.246997    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247358 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247101    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247486 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247450    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247630 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247511    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.247630 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.247549    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.249316 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.249271    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.249574 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.249555    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.254771 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.254748    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9xv\" (UniqueName: \"kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv\") pod \"rate-test-model-3ee067-kserve-79f8745d7f-9w27h\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") " pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.412128 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.412093    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:15.587997 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.587958    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:15.753689 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.753665    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:37:15.755664 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:37:15.755632    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddf8af17_4857_4d1f_be79_2d3fe97c93e0.slice/crio-f40584d78ba7f78ee15d0c1d9870f87df11133bbc7709501a61a3df2bebeffae WatchSource:0}: Error finding container f40584d78ba7f78ee15d0c1d9870f87df11133bbc7709501a61a3df2bebeffae: Status 404 returned error can't find the container with id f40584d78ba7f78ee15d0c1d9870f87df11133bbc7709501a61a3df2bebeffae
Jul 07 00:37:15.836556 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.836519    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" event={"ID":"ddf8af17-4857-4d1f-be79-2d3fe97c93e0","Type":"ContainerStarted","Data":"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"}
Jul 07 00:37:15.836722 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.836570    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" event={"ID":"ddf8af17-4857-4d1f-be79-2d3fe97c93e0","Type":"ContainerStarted","Data":"f40584d78ba7f78ee15d0c1d9870f87df11133bbc7709501a61a3df2bebeffae"}
Jul 07 00:37:15.858783 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:15.858735    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" podStartSLOduration=0.858721491 podStartE2EDuration="858.721491ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:37:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:37:15.855884636 +0000 UTC m=+2421.180887945" watchObservedRunningTime="2026-07-07 00:37:15.858721491 +0000 UTC m=+2421.183724775"
Jul 07 00:37:16.167035 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:16.167006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:25.412987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:25.412899    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:25.412987 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:25.412956    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:25.425285 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:25.425257    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:25.904578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:25.904552    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:37:53.527689 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:53.527590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:54.373592 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:54.373549    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:37:54.392179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:54.392147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:55.498200 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:55.498161    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:55.939726 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:55.939690    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:56.040261 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:56.040226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:56.145885 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:56.145845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:37:56.280434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:37:56.280351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:01.012216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:01.012181    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:09.393723 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.393677    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" podUID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" containerName="main" containerID="cri-o://17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9" gracePeriod=45
Jul 07 00:38:09.658610 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.658542    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:38:09.769767 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769714    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.769767 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769771    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.770054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769799    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.770054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769852    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt9xv\" (UniqueName: \"kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.770054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769889    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.770054 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.769936    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir\") pod \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\" (UID: \"ddf8af17-4857-4d1f-be79-2d3fe97c93e0\") "
Jul 07 00:38:09.770283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.770055    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:09.770283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.770225    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home" (OuterVolumeSpecName: "home") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:09.770283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.770263    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-model-cache\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:09.770283 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.770267    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:09.771935 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.771912    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm" (OuterVolumeSpecName: "dshm") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:09.772041 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.772020    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv" (OuterVolumeSpecName: "kube-api-access-vt9xv") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "kube-api-access-vt9xv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:38:09.772092 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.772035    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddf8af17-4857-4d1f-be79-2d3fe97c93e0" (UID: "ddf8af17-4857-4d1f-be79-2d3fe97c93e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 07 00:38:09.871080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.871040    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-dshm\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:09.871080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.871075    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tls-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:09.871080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.871086    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt9xv\" (UniqueName: \"kubernetes.io/projected/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-kube-api-access-vt9xv\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:09.871080 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.871094    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-home\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:09.871398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:09.871103    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddf8af17-4857-4d1f-be79-2d3fe97c93e0-tmp-dir\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:10.114428 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.114330    2583 generic.go:356] "Generic (PLEG): container finished" podID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" containerID="17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9" exitCode=0
Jul 07 00:38:10.114428 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.114376    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"
Jul 07 00:38:10.114652 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.114419    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" event={"ID":"ddf8af17-4857-4d1f-be79-2d3fe97c93e0","Type":"ContainerDied","Data":"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"}
Jul 07 00:38:10.114652 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.114462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h" event={"ID":"ddf8af17-4857-4d1f-be79-2d3fe97c93e0","Type":"ContainerDied","Data":"f40584d78ba7f78ee15d0c1d9870f87df11133bbc7709501a61a3df2bebeffae"}
Jul 07 00:38:10.114652 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.114479    2583 scope.go:117] "RemoveContainer" containerID="17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"
Jul 07 00:38:10.125533 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.125482    2583 scope.go:117] "RemoveContainer" containerID="17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"
Jul 07 00:38:10.126089 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:38:10.126069    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9\": container with ID starting with 17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9 not found: ID does not exist" containerID="17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"
Jul 07 00:38:10.126174 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.126097    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9"} err="failed to get container status \"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9\": rpc error: code = NotFound desc = could not find container \"17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9\": container with ID starting with 17487f4ff5e176469b602ca338173f187dca208a5399d8d6817cce9113d3c9e9 not found: ID does not exist"
Jul 07 00:38:10.138202 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.138172    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:38:10.141648 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:10.141622    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3ee067/rate-test-model-3ee067-kserve-79f8745d7f-9w27h"]
Jul 07 00:38:11.157865 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:11.157831    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" path="/var/lib/kubelet/pods/ddf8af17-4857-4d1f-be79-2d3fe97c93e0/volumes"
Jul 07 00:38:13.523195 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:13.523158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:13.625149 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:13.625113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:21.283883 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:21.283848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:21.545463 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:21.545373    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:38:21.545685 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:21.545638    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy" containerID="cri-o://59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389" gracePeriod=30
Jul 07 00:38:21.558781 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:21.558754    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:22.001627 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:22.001594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:22.427078 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:22.427044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:22.488235 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:22.488200    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:26.808725 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.808666    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:38:26.837664 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837633    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdlfj\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837685    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837716    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837749    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837773    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837797    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.837854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837843    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.838179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837870    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.838179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.837922    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert\") pod \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\" (UID: \"2877e0ec-e4ec-4299-9e90-073ab11e00c4\") "
Jul 07 00:38:26.838179 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838163    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:26.838350 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838177    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:26.838350 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838261    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.838350 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838282    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.838520 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838482    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:38:26.839096 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.838892    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:26.839096 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.839060    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:26.840529 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.840501    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj" (OuterVolumeSpecName: "kube-api-access-tdlfj") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "kube-api-access-tdlfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:38:26.840679 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.840648    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:38:26.840794 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.840767    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:38:26.840881 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.840859    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2877e0ec-e4ec-4299-9e90-073ab11e00c4" (UID: "2877e0ec-e4ec-4299-9e90-073ab11e00c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:26.939606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939563    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939602    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939611    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939618    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939627    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdlfj\" (UniqueName: \"kubernetes.io/projected/2877e0ec-e4ec-4299-9e90-073ab11e00c4-kube-api-access-tdlfj\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939637    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:26.939887 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:26.939645    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2877e0ec-e4ec-4299-9e90-073ab11e00c4-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:27.198491 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.198458    2583 generic.go:356] "Generic (PLEG): container finished" podID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerID="59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389" exitCode=0
Jul 07 00:38:27.198671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.198503    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"
Jul 07 00:38:27.198671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.198532    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerDied","Data":"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"}
Jul 07 00:38:27.198671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.198584    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs" event={"ID":"2877e0ec-e4ec-4299-9e90-073ab11e00c4","Type":"ContainerDied","Data":"78ee947ad385fe902e854f8143c595a3f8d354e8f0114d05760dea784252ea8d"}
Jul 07 00:38:27.198671 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.198605    2583 scope.go:117] "RemoveContainer" containerID="59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"
Jul 07 00:38:27.207499 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.207478    2583 scope.go:117] "RemoveContainer" containerID="8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"
Jul 07 00:38:27.215313 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.215274    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:38:27.215665 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.215651    2583 scope.go:117] "RemoveContainer" containerID="59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"
Jul 07 00:38:27.215958 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:38:27.215937    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389\": container with ID starting with 59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389 not found: ID does not exist" containerID="59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"
Jul 07 00:38:27.216039 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.215967    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389"} err="failed to get container status \"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389\": rpc error: code = NotFound desc = could not find container \"59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389\": container with ID starting with 59b4956506faa9652e00e79647c0d5f8355e498fa9022a1a97b7f2271ca0f389 not found: ID does not exist"
Jul 07 00:38:27.216039 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.215987    2583 scope.go:117] "RemoveContainer" containerID="8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"
Jul 07 00:38:27.216275 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:38:27.216242    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff\": container with ID starting with 8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff not found: ID does not exist" containerID="8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"
Jul 07 00:38:27.216436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.216277    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff"} err="failed to get container status \"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff\": rpc error: code = NotFound desc = could not find container \"8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff\": container with ID starting with 8e88402106da44359b12124192b24e1aef45f04558e46202fdd7f504cb23d6ff not found: ID does not exist"
Jul 07 00:38:27.218946 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:27.218925    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-a8caec-openshift-default-65bc5f994c-nbffs"]
Jul 07 00:38:28.357433 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:28.357399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:28.456415 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:28.456374    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:28.619600 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:28.619569    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:38:28.619877 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:28.619849    2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy" containerID="cri-o://04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962" gracePeriod=30
Jul 07 00:38:28.637650 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:28.637618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:29.158409 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:29.158372    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" path="/var/lib/kubelet/pods/2877e0ec-e4ec-4299-9e90-073ab11e00c4/volumes"
Jul 07 00:38:29.735879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:29.735843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:33.881742 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:33.881678    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:38:34.009714 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009676    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009728    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009744    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009772    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7ckh\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009809    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009864    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.009907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009897    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.010216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009929    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.010216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.009968    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token\") pod \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\" (UID: \"0cd93be0-37e8-4c46-960f-a7c34bd8e837\") "
Jul 07 00:38:34.010216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010016    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:34.010216 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010062    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:34.010465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010211    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:34.010465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010268    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.010465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010283    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-workload-certs\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.010465 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010330    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 07 00:38:34.010800 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.010760    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data" (OuterVolumeSpecName: "istio-data") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:34.012177 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.012150    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 07 00:38:34.012321 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.012238    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token" (OuterVolumeSpecName: "istio-token") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:38:34.012554 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.012538    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 07 00:38:34.012619 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.012566    2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh" (OuterVolumeSpecName: "kube-api-access-n7ckh") pod "0cd93be0-37e8-4c46-960f-a7c34bd8e837" (UID: "0cd93be0-37e8-4c46-960f-a7c34bd8e837"). InnerVolumeSpecName "kube-api-access-n7ckh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 07 00:38:34.111666 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111625    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-credential-socket\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111666 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111659    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istiod-ca-cert\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111666 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111668    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-envoy\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111666 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111676    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-token\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111683    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-data\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111691    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7ckh\" (UniqueName: \"kubernetes.io/projected/0cd93be0-37e8-4c46-960f-a7c34bd8e837-kube-api-access-n7ckh\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.111942 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.111699    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cd93be0-37e8-4c46-960f-a7c34bd8e837-istio-podinfo\") on node \"ip-10-0-142-92.ec2.internal\" DevicePath \"\""
Jul 07 00:38:34.150798 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.150726    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2vdcm_3e915ee8-6de5-4fd8-90cb-86e82fad7377/manager/0.log"
Jul 07 00:38:34.234032 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.233995    2583 generic.go:356] "Generic (PLEG): container finished" podID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerID="04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962" exitCode=0
Jul 07 00:38:34.234180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.234040    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"
Jul 07 00:38:34.234180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.234068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerDied","Data":"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"}
Jul 07 00:38:34.234180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.234116    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf" event={"ID":"0cd93be0-37e8-4c46-960f-a7c34bd8e837","Type":"ContainerDied","Data":"7b212a78c5cdb31a329dcbecfd50611613bfbca8a3ed8b7170e0cba1089e5a1d"}
Jul 07 00:38:34.234180 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.234139    2583 scope.go:117] "RemoveContainer" containerID="04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"
Jul 07 00:38:34.243813 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.243790    2583 scope.go:117] "RemoveContainer" containerID="471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"
Jul 07 00:38:34.252099 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.252083    2583 scope.go:117] "RemoveContainer" containerID="04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"
Jul 07 00:38:34.252374 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:38:34.252354    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962\": container with ID starting with 04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962 not found: ID does not exist" containerID="04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"
Jul 07 00:38:34.252437 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.252383    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962"} err="failed to get container status \"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962\": rpc error: code = NotFound desc = could not find container \"04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962\": container with ID starting with 04eb8c80ebd388fbfeb5d24482fc6734f915afacac5b58c8e436611177fc4962 not found: ID does not exist"
Jul 07 00:38:34.252437 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.252400    2583 scope.go:117] "RemoveContainer" containerID="471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"
Jul 07 00:38:34.252642 ip-10-0-142-92 kubenswrapper[2583]: E0707 00:38:34.252624    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd\": container with ID starting with 471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd not found: ID does not exist" containerID="471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"
Jul 07 00:38:34.252681 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.252648    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd"} err="failed to get container status \"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd\": rpc error: code = NotFound desc = could not find container \"471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd\": container with ID starting with 471bc84d87a0238314b3c28f9d045d4fe7eb5f4620ee2b97273be0cfd8364bdd not found: ID does not exist"
Jul 07 00:38:34.257106 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.257084    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:38:34.261398 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.261377    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-3ee067-openshift-default-5b95c9b4f5-twdbf"]
Jul 07 00:38:34.268271 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.268251    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2lbc5_a5b59986-fc18-4fd7-8bd9-ec3803d58fa0/manager/0.log"
Jul 07 00:38:34.535914 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.535837    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kqksn_31a32a62-84c5-4cec-9b52-ee28f8bfddb3/server/0.log"
Jul 07 00:38:34.662463 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:34.662436    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9d52g_d53f54a4-54a3-405d-8987-a4ca486b9991/manager/0.log"
Jul 07 00:38:35.048537 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:35.048512    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-zhxp2_196299b4-a3e0-4c6f-92c5-d3125246886c/copy-manifests/0.log"
Jul 07 00:38:35.061473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:35.061441    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-zhxp2_196299b4-a3e0-4c6f-92c5-d3125246886c/manager/0.log"
Jul 07 00:38:35.157862 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:35.157833    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" path="/var/lib/kubelet/pods/0cd93be0-37e8-4c46-960f-a7c34bd8e837/volumes"
Jul 07 00:38:36.560108 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:36.560075    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9bjnd_065d5038-8b3b-4f7b-824e-5ddd409e6e4b/manager/0.log"
Jul 07 00:38:36.683395 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:36.683357    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-df5h6_a98f2b02-4133-41a9-8fe9-d07782183190/manager/0.log"
Jul 07 00:38:36.926256 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:36.926224    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-99gsc_5f46530c-483c-4e73-92a6-c3922ab9227e/registry-server/0.log"
Jul 07 00:38:37.060155 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:37.060126    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-29nx5_0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b/manager/0.log"
Jul 07 00:38:37.321318 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:37.321211    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-q92m4_093daf68-6461-4c4e-acad-f643ee39b22d/manager/0.log"
Jul 07 00:38:37.693526 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:37.693496    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4khb4s_42796cd9-a85e-40ea-9d1a-4bd12addb356/istio-proxy/0.log"
Jul 07 00:38:38.058177 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:38.058098    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-568f9c9968-m68pf_f65a97a0-2704-48f5-a88a-6f8e8b32cf2d/kube-auth-proxy/0.log"
Jul 07 00:38:38.175496 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:38.175459    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/1.log"
Jul 07 00:38:38.552971 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:38.552937    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fb4c889c7-cwvmd_e82e9c05-b03b-461b-8959-3229ba134a61/router/0.log"
Jul 07 00:38:39.046597 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:39.046569    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hdnq7_da3a59c2-3fa1-403b-8717-a364d7824792/main/0.log"
Jul 07 00:38:39.287888 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:39.287852    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbbv8sp_c6bb2b2f-9ae5-4ff5-b3c9-1bfc78c33cd5/main/0.log"
Jul 07 00:38:44.125000 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.124967    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7rzff/must-gather-k5k2k"]
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125348    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125360    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125367    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" containerName="main"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125374    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" containerName="main"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125385    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125392    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125402    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125408    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125414    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.125416 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125418    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.125731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125483    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125490    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.125731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125498    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddf8af17-4857-4d1f-be79-2d3fe97c93e0" containerName="main"
Jul 07 00:38:44.125731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125608    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd93be0-37e8-4c46-960f-a7c34bd8e837" containerName="istio-proxy"
Jul 07 00:38:44.125731 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.125614    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="2877e0ec-e4ec-4299-9e90-073ab11e00c4" containerName="istio-proxy"
Jul 07 00:38:44.129952 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.129929    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.132874 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.132851    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rzff\"/\"kube-root-ca.crt\""
Jul 07 00:38:44.132874 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.132859    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7rzff\"/\"default-dockercfg-h4m67\""
Jul 07 00:38:44.133040 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.132858    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7rzff\"/\"openshift-service-ca.crt\""
Jul 07 00:38:44.137436 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.137415    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/must-gather-k5k2k"]
Jul 07 00:38:44.148907 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.148879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:44.204970 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.204934    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a34c075f-311f-4332-9e8c-878cd4686344-must-gather-output\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.204970 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.204971    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv8gn\" (UniqueName: \"kubernetes.io/projected/a34c075f-311f-4332-9e8c-878cd4686344-kube-api-access-kv8gn\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.306051 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.306017    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a34c075f-311f-4332-9e8c-878cd4686344-must-gather-output\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.306051 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.306054    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv8gn\" (UniqueName: \"kubernetes.io/projected/a34c075f-311f-4332-9e8c-878cd4686344-kube-api-access-kv8gn\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.306449 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.306411    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a34c075f-311f-4332-9e8c-878cd4686344-must-gather-output\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.315403 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.315379    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv8gn\" (UniqueName: \"kubernetes.io/projected/a34c075f-311f-4332-9e8c-878cd4686344-kube-api-access-kv8gn\") pod \"must-gather-k5k2k\" (UID: \"a34c075f-311f-4332-9e8c-878cd4686344\") " pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.439831 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.439740    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/must-gather-k5k2k"
Jul 07 00:38:44.571916 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.571891    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/must-gather-k5k2k"]
Jul 07 00:38:44.574384 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:38:44.574353    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda34c075f_311f_4332_9e8c_878cd4686344.slice/crio-5cde6e37bbe30d742dd34b8a2dff6814406362f1866c92401e086008c3db0ba2 WatchSource:0}: Error finding container 5cde6e37bbe30d742dd34b8a2dff6814406362f1866c92401e086008c3db0ba2: Status 404 returned error can't find the container with id 5cde6e37bbe30d742dd34b8a2dff6814406362f1866c92401e086008c3db0ba2
Jul 07 00:38:44.576473 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:44.576459    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 07 00:38:45.305657 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:45.305611    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/must-gather-k5k2k" event={"ID":"a34c075f-311f-4332-9e8c-878cd4686344","Type":"ContainerStarted","Data":"5cde6e37bbe30d742dd34b8a2dff6814406362f1866c92401e086008c3db0ba2"}
Jul 07 00:38:45.472083 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:45.472047    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:38:46.313237 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:46.313198    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/must-gather-k5k2k" event={"ID":"a34c075f-311f-4332-9e8c-878cd4686344","Type":"ContainerStarted","Data":"e08af7e635c75f2c331e38ac324e751eede66b2c619b5dee494643200d5b6f46"}
Jul 07 00:38:46.313764 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:46.313251    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/must-gather-k5k2k" event={"ID":"a34c075f-311f-4332-9e8c-878cd4686344","Type":"ContainerStarted","Data":"8a8c47872332c38814d06754ceff3c31c852ffbf6b4062a2f812295800b8b69b"}
Jul 07 00:38:46.334406 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:46.334339    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7rzff/must-gather-k5k2k" podStartSLOduration=1.536811433 podStartE2EDuration="2.334316523s" totalImagesPullingTime="797.50509ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:38:46.332025659 +0000 UTC m=+2511.657028943" watchObservedRunningTime="2026-07-07 00:38:46.334316523 +0000 UTC m=+2511.659319807"
Jul 07 00:38:47.149566 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:47.149495    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hxjb2_534cd6f5-0f18-4837-90e0-e83f0a16cefe/global-pull-secret-syncer/0.log"
Jul 07 00:38:47.308049 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:47.308018    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log"
Jul 07 00:38:47.465191 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:47.465106    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-92.ec2.internal_936b88c7565d9c88f04260646b598913/haproxy/0.log"
Jul 07 00:38:52.393168 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:52.393131    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9bjnd_065d5038-8b3b-4f7b-824e-5ddd409e6e4b/manager/0.log"
Jul 07 00:38:52.429939 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:52.429904    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-df5h6_a98f2b02-4133-41a9-8fe9-d07782183190/manager/0.log"
Jul 07 00:38:52.500224 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:52.500186    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-99gsc_5f46530c-483c-4e73-92a6-c3922ab9227e/registry-server/0.log"
Jul 07 00:38:52.680983 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:52.680863    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-29nx5_0adbe9f5-f9f6-48fd-9f21-42a1687c8e7b/manager/0.log"
Jul 07 00:38:52.783929 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:52.783862    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-q92m4_093daf68-6461-4c4e-acad-f643ee39b22d/manager/0.log"
Jul 07 00:38:54.608434 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:54.608402    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-nc99n_a1cc70ee-c768-4fbc-b89e-d6e977fe751c/cluster-monitoring-operator/0.log"
Jul 07 00:38:54.851377 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:54.851257    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv95b_3985d385-b607-4e6e-9c97-cca5fdca768f/node-exporter/0.log"
Jul 07 00:38:54.872919 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:54.872871    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv95b_3985d385-b607-4e6e-9c97-cca5fdca768f/kube-rbac-proxy/0.log"
Jul 07 00:38:54.895797 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:54.895759    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rv95b_3985d385-b607-4e6e-9c97-cca5fdca768f/init-textfile/0.log"
Jul 07 00:38:57.243266 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.243228    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"]
Jul 07 00:38:57.247900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.247872    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.256269 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.256231    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"]
Jul 07 00:38:57.270566 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.270534    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-proc\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.270854 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.270832    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-lib-modules\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.271018 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.271003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-sys\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.271162 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.271149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-podres\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.271360 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.271344    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wk7m\" (UniqueName: \"kubernetes.io/projected/637c22a7-588a-48e7-aee6-6203b0048873-kube-api-access-4wk7m\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.367492 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.367460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/1.log"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.371918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-proc\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.371972    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-lib-modules\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372011    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-sys\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372044    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-podres\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372066    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-proc\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372123    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wk7m\" (UniqueName: \"kubernetes.io/projected/637c22a7-588a-48e7-aee6-6203b0048873-kube-api-access-4wk7m\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372129    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-sys\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372196    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-lib-modules\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.372446 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.372198    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/637c22a7-588a-48e7-aee6-6203b0048873-podres\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.375155 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.375120    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-fbxck_05d4aaed-3119-474c-acc7-b981e197aeb0/console-operator/2.log"
Jul 07 00:38:57.380606 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.380581    2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wk7m\" (UniqueName: \"kubernetes.io/projected/637c22a7-588a-48e7-aee6-6203b0048873-kube-api-access-4wk7m\") pod \"perf-node-gather-daemonset-kllx8\" (UID: \"637c22a7-588a-48e7-aee6-6203b0048873\") " pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.562841 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.562724    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:57.754085 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.754036    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"]
Jul 07 00:38:57.758871 ip-10-0-142-92 kubenswrapper[2583]: W0707 00:38:57.758836    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod637c22a7_588a_48e7_aee6_6203b0048873.slice/crio-69d07acf2a0145ed1f7642104398d42b707eae1d41d7c60bfe30224d6bbabe7e WatchSource:0}: Error finding container 69d07acf2a0145ed1f7642104398d42b707eae1d41d7c60bfe30224d6bbabe7e: Status 404 returned error can't find the container with id 69d07acf2a0145ed1f7642104398d42b707eae1d41d7c60bfe30224d6bbabe7e
Jul 07 00:38:57.941697 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:57.941670    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-79f496b6c-j897w_99c40444-d40a-4890-bf47-9e2929fb28ca/console/0.log"
Jul 07 00:38:58.396930 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:58.396892    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8" event={"ID":"637c22a7-588a-48e7-aee6-6203b0048873","Type":"ContainerStarted","Data":"702140f5f5f0d9349cd52c4c9270c7eb16e34fd088efc45b0b3b020743e1909d"}
Jul 07 00:38:58.397364 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:58.396962    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:38:58.397364 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:58.396985    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8" event={"ID":"637c22a7-588a-48e7-aee6-6203b0048873","Type":"ContainerStarted","Data":"69d07acf2a0145ed1f7642104398d42b707eae1d41d7c60bfe30224d6bbabe7e"}
Jul 07 00:38:58.430552 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:58.430490    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8" podStartSLOduration=1.430472986 podStartE2EDuration="1.430472986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:38:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:38:58.42701778 +0000 UTC m=+2523.752021064" watchObservedRunningTime="2026-07-07 00:38:58.430472986 +0000 UTC m=+2523.755476271"
Jul 07 00:38:58.592599 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:58.592565    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-2xc2q_16821c74-4f6c-4568-bb61-8acbab454769/volume-data-source-validator/0.log"
Jul 07 00:38:59.643238 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:59.643206    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qrhjm_9745e43d-f62d-476a-980f-54ad7ccf8c8f/dns/0.log"
Jul 07 00:38:59.664069 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:59.664036    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qrhjm_9745e43d-f62d-476a-980f-54ad7ccf8c8f/kube-rbac-proxy/0.log"
Jul 07 00:38:59.736735 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:38:59.736702    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-crvxn_579e3da3-6a24-4a7d-a719-519db369ad56/dns-node-resolver/0.log"
Jul 07 00:39:00.403879 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:00.403850    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c5v2z_d32f0935-e944-44ed-a4c6-2f0e8c58376d/node-ca/0.log"
Jul 07 00:39:01.336092 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:01.336052    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4khb4s_42796cd9-a85e-40ea-9d1a-4bd12addb356/istio-proxy/0.log"
Jul 07 00:39:01.592736 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:01.592660    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-568f9c9968-m68pf_f65a97a0-2704-48f5-a88a-6f8e8b32cf2d/kube-auth-proxy/0.log"
Jul 07 00:39:01.622162 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:01.622120    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/0.log"
Jul 07 00:39:01.659891 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:01.659852    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/1.log"
Jul 07 00:39:01.799663 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:01.799634    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fb4c889c7-cwvmd_e82e9c05-b03b-461b-8959-3229ba134a61/router/0.log"
Jul 07 00:39:02.503776 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:02.503748    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ccxln_e2f224ba-1efd-4b24-b548-44b7c311c4f3/serve-healthcheck-canary/0.log"
Jul 07 00:39:02.992381 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:02.992286    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ztxlq_399f91a1-6840-42c2-b390-bb3e18779244/insights-operator/0.log"
Jul 07 00:39:02.995164 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:02.995121    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-ztxlq_399f91a1-6840-42c2-b390-bb3e18779244/insights-operator/1.log"
Jul 07 00:39:03.012924 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:03.012893    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:39:03.068674 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:03.068640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:39:03.155310 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:03.155254    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgr8w_9918fb3f-6f60-4983-94fb-7c5ef286f0d0/kube-rbac-proxy/0.log"
Jul 07 00:39:03.176509 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:03.176473    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgr8w_9918fb3f-6f60-4983-94fb-7c5ef286f0d0/exporter/0.log"
Jul 07 00:39:03.196081 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:03.196053    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zgr8w_9918fb3f-6f60-4983-94fb-7c5ef286f0d0/extractor/0.log"
Jul 07 00:39:04.417098 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:04.416242    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7rzff/perf-node-gather-daemonset-kllx8"
Jul 07 00:39:05.255241 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.255201    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2vdcm_3e915ee8-6de5-4fd8-90cb-86e82fad7377/manager/0.log"
Jul 07 00:39:05.438914 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.438854    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-2lbc5_a5b59986-fc18-4fd7-8bd9-ec3803d58fa0/manager/0.log"
Jul 07 00:39:05.622157 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.622129    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kqksn_31a32a62-84c5-4cec-9b52-ee28f8bfddb3/server/0.log"
Jul 07 00:39:05.711652 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.711618    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-9d52g_d53f54a4-54a3-405d-8987-a4ca486b9991/manager/0.log"
Jul 07 00:39:05.913480 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.913380    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-zhxp2_196299b4-a3e0-4c6f-92c5-d3125246886c/manager/0.log"
Jul 07 00:39:05.931226 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:05.931195    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b6d6b477c-zhxp2_196299b4-a3e0-4c6f-92c5-d3125246886c/copy-manifests/0.log"
Jul 07 00:39:07.869022 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:07.868988    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4khb4s_42796cd9-a85e-40ea-9d1a-4bd12addb356/istio-proxy/0.log"
Jul 07 00:39:08.110889 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:08.110860    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-568f9c9968-m68pf_f65a97a0-2704-48f5-a88a-6f8e8b32cf2d/kube-auth-proxy/0.log"
Jul 07 00:39:08.146726 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:08.146646    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/0.log"
Jul 07 00:39:08.180994 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:08.180951    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/1.log"
Jul 07 00:39:08.316753 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:08.316709    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fb4c889c7-cwvmd_e82e9c05-b03b-461b-8959-3229ba134a61/router/0.log"
Jul 07 00:39:09.507464 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:09.507395    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4khb4s_42796cd9-a85e-40ea-9d1a-4bd12addb356/istio-proxy/0.log"
Jul 07 00:39:09.733941 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:09.733914    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-568f9c9968-m68pf_f65a97a0-2704-48f5-a88a-6f8e8b32cf2d/kube-auth-proxy/0.log"
Jul 07 00:39:09.756448 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:09.756404    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/0.log"
Jul 07 00:39:09.795838 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:09.795758    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-trhm5_f826f9ad-2195-4107-b8e4-3d7ecee2ff27/istio-proxy/1.log"
Jul 07 00:39:09.914178 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:09.914140    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5fb4c889c7-cwvmd_e82e9c05-b03b-461b-8959-3229ba134a61/router/0.log"
Jul 07 00:39:12.977537 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:12.977506    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-4ckw6_699771fc-b797-47a3-9c37-81b0ebbfb0dd/kube-storage-version-migrator-operator/1.log"
Jul 07 00:39:12.978339 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:12.978321    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-4ckw6_699771fc-b797-47a3-9c37-81b0ebbfb0dd/kube-storage-version-migrator-operator/0.log"
Jul 07 00:39:13.477711 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:13.477667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:39:13.531631 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:13.531565    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bbcqn_bf55712a-27f6-4122-9d81-349420350fb2/konnectivity-agent/0.log" limit=1024
Jul 07 00:39:14.002488 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.002460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/kube-multus-additional-cni-plugins/0.log"
Jul 07 00:39:14.025500 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.025472    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/egress-router-binary-copy/0.log"
Jul 07 00:39:14.047185 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.047161    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/cni-plugins/0.log"
Jul 07 00:39:14.068646 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.068621    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/bond-cni-plugin/0.log"
Jul 07 00:39:14.090275 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.090254    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/routeoverride-cni/0.log"
Jul 07 00:39:14.112578 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.112556    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/whereabouts-cni-bincopy/0.log"
Jul 07 00:39:14.134059 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.133999    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zdcz_88bbc328-c0d6-415d-957a-b3d9a9c65aba/whereabouts-cni/0.log"
Jul 07 00:39:14.496900 ip-10-0-142-92 kubenswrapper[2583]: I0707 00:39:14.496817    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-knjcv_11671568-9dd3-45a0-825d-3b1a2c75234a/kube-multus/0.log"